Digital Twins and the New Reality

Imagine this: You’re a designer working for NASA, a member of the moon colonization project team. The solar array responsible for keeping the base operational just went down. You stroll onto the lunar landscape, remove the main control panel and quickly determine there’s a problem with the DC converter. After verifying the device’s serial number and checking the schematics, you’re able to bring the unit back online. You’re a space hero!

Closer to Home

Not really. You performed this heroic feat from mission control in Houston. Your eyes were a mixed reality (MR) headset, your hands those of a spacefaring repair robot, the DC converter a digital twin of the actual device sitting 238,000 miles away. And while you’re an excellent engineer, you’re no expert on this particular system—it was the detailed information contained within the digital twin and overlaid on the video feed of the physical device that saved the day, not your skills as an electrical technician. 

The ability to operate robots remotely is one of many tasks made possible with digital twins and mixed reality. Image courtesy of Autodesk.

This futuristic event isn’t as farfetched as one might think. Nicholas Cote, Autodesk robotics researcher suggested a similar scenario while speaking at a 2017 Autodesk University event, but used Mars and the International Space Station as examples. His colleague, Senior Research Engineer Heather Kerrick, offered several terrestrial reasons for combining digital twins with robotics and MR systems, with safer, more efficient workplaces among them. 

“For as long as there’ve been robots, it’s taken a lot of time and expense and a certain degree of risk to move them around,” Kerrick says. “Mixed reality will serve to reduce these challenges.”

Reality? What Reality?

Exactly what is MR, and how is it different from virtual reality (VR) and augmented reality (AR)?

“With VR, you wear a set of goggles that completely occludes you from the real world, transporting yourself via time or scale or distance or environment to somewhere else, a virtual place that exists only for you,” explains Mike Campbell, executive vice president and head of products for the AR business at PTC. “On the other end of the spectrum there’s AR, which some refer to as assisted reality—the heads-up display in a military aircraft, for instance, or Google Glass, both of which overlay a digital image on top of a physical object.”

MR combines the best of both virtual worlds, says Campbell. Similar to VR, it typically employs a binocular device that covers the user’s entire field of view. Unlike VR, however, an MR user is very much able to see and interact with the physical world while viewing the 3D digital content placed atop it. 

If you were to view a tractor in the MR world, for example, its digital twin would be aligned with and anchored to the physical object. Assuming the digital twin contains the requisite information, you could learn how to repair the tractor, order parts for it or understand how and where it was manufactured, all from within the MR landscape. It might not be as fun as riding a virtual rollercoaster or flying a fighter jet, but it’s close. 

PTC customer Howden has realized significant benefits from its mixed reality-digital twin initiative. Image courtesy of PTC.

Campbell agrees. “One of our customers, Howden, is a manufacturer of compressors, heat exchangers, steam turbines and other rotating equipment, and has made its products IIoT (Industrial Internet of Things) capable,” he says. “They use ThingWorx to gather end-use performance data, Vuforia Studio to create digital content, then apply both to digital twins of their systems. Now, an operator or service technician can use MR to view and interact with the equipment—they can push virtual buttons to view operational values like air temperature or oil pressure, view animated work instructions, update the digital twin with relevant field data and much more. This is why we consider MR to be the next generation human-machine interface.” 

Opening Doors

Think about the possibilities: Even without a moon-roving robot at the other end, MR opens doors no one knew existed even a decade ago. The training potential is enormous, giving less-skilled workers the means to operate machinery or assemble products they’ve never seen. It lets designers in Ohio visit their supplier’s factory floor in Poughkeepsie, New York, or show potential customers in Shanghai their latest and greatest creations, both without leaving the office. And collaboration? You can forget those traditional WebEx meetings—simply don a HoloLens 2 or similar MR device and you’ll soon be having show-and-tell sessions that people actually want to attend. 

Mixed reality makes it easy to ask for help if users are unsure how to assemble a widget or fix a doodad. It also makes it easy to order parts or update a workorder, through integration to leading ERP systems such as Dynamics 365. Image courtesy of Microsoft.

The Microsoft HoloLens has several real-world examples of MR use. BAE Systems is working with PTC to create mixed reality solutions for the electric propulsion systems used on its HybriDrive buses, dramatically improving worker efficiency while reducing training time by up to 40%. 

Global positioning technology provider Trimble is said to have transformed its solutions for the construction and mining industries using MR. The Synchro 4D construction management software developed by Bentley Systems is now integrated with HoloLens 2 technology, which enables users to visualize real-time project updates directly over project sites and models.

“At Microsoft, we believe that mixed reality is a key part of the future of computing,” says Terry Farrell, director of Mixed Reality Marketing at Microsoft. “We have seen early adoption in the enterprise across several industries and in organizations like Ford, Chevron and thyssenkrupp, for a variety of applications from remote assistance and training to data visualization. Microsoft HoloLens has proven to be a useful tool, and we believe we are only at the beginning of what we can enable for our customers and partners.” 

Consummating the Marriage

Another company with its eye on the virtual prize is Siemens PLM Software. Senior Technical Architect Ian Fisher’s view of the digital twin is that people are able to learn from it, gain valuable insights that help them to make informed and data-driven decisions, and simultaneously enhance the digital twin with real-world information—augmented and virtual reality is the tool that sits between the two, essentially helping people to complete the marriage of the virtual and physical worlds and improve both. 

To that end, Siemens PLM is taking steps to make MR an integral part of its entire product suite. The HoloLens and other brands of MR headsets will play a role, but Fisher is quick to note that customers shouldn’t be limited to what are—at least for the time being—fairly expensive devices. Though not as all-encompassing, smartphones, tablets and other mobile devices will also serve as valuable windows into the MR realm, putting it within reach of even the smaller design firms and manufacturers. 

“The digital twin was already a very exciting trend for us and others in this industry, but the addition of easily adoptable augmented and virtual reality tools is taking it to an entirely different level,” Fisher says. 

A wind turbine inspector can use MR to view the digital twin out in the field, for instance, and note that a certain access panel was difficult to remove, and provide immediate feedback to the designer with the make, model and location of the equipment in question. 

Or, a service technician might find an installation fault; using MR, they can log an issue in real time that the work instructions attached to a machine’s digital twin were incorrect—after updating them, the factory supervisor could be notified, avoiding future faults and preventing a production stoppage. 

Traditionally, the information in both of these instances and countless others had to be conveyed by telephone or email, or not at all. Now, it’s a permanent part of the digital twin. 

“There are so many fun examples out there where the digital twin together with augmented reality are making every aspect of the product lifecycle far easier to manage,” Fisher says. “We as an industry are really just getting started, but are already seeing significant benefits for all involved. It’s going to affect virtually everything we can imagine.”