Future Forward Full Interview: Innovative Mixed-Reality Technology Improves Construction Monitoring
This page profiles innovative and impactful applied research and development in civil and structural engineering to spur continuing thought and dialog to create a better industry. These profiles are based on interviews, and the opinions and statements are those of the subject and are not necessarily shared or endorsed by this publication.
This particular interview was recorded during Bentley Systems’ Year In Infrastructure 2018 Event in London, where Nancy Li was a guest speaker. Click this link to view a video of the interview.
Nancy Li is the Global Head of Partnerships and Industry Solutions for Microsoft’s Mixed Reality and AI Perception group.
V1 Media: Can you describe your current role at Microsoft and what you do there?
Li: I’m from the product engineering team for Mixed Reality and AI Perceptions from Microsoft, and I drive our global industry solutions portfolio. We have teams around the world working with solution partners and system integrators to enable our costumers across all the high-signal industries to start using HoloLens and all the technology in that realm.
V1 Media: Tell me a little bit about what brought you to this Bentley conference?
Li: We’ve been working with Bentley Systems and specifically with Synchro for quite a while, and we started a conversation about them building mixed-reality solutions two years ago. And we started experimenting, and we started building and commercializing the mixed-reality solutions to enable people to view 3D architectural CAD models onsite to be able to do much more efficient and accurate construction monitoring, construction inspection and things like that. And as we started with Synchro, which is a 4D solution, we’re also now discovering opportunities. And Bentley’s costumers are asking us to be able to integrate Bentley models into many different kinds of mixed-reality solutions. So that’s kind of the wider and longer term and close relationship in the partnership we have with Bentley. And that’s what brought us here.
V1 Media: Can you tell me about one of these customers or solutions that you’re working with specifically?
Li: We worked with FC Barcelona. FC Barcelona is rebuilding their stadium, and the difficulty in this process is that FC Barcelona–during the entire gigantic construction–every single weekend they still have to welcome a hundred thousand people to come into the stadium to watch the games while the entire construction is going on. And so that requires accurate construction monitoring, which the existing technology does not enable. So then they looked to us and they looked to Bentley to help them create a construction monitoring solution. Essentially what we did is we take the IoT camera information of the construction site, overlaying it with the Bentley 4D models on top. And this enables remote monitoring with our Microsoft computer-vision service and doing a lot of accurate alignment to be able to do remote construction monitoring.
And at the same time, we have also built the HoloLens onsite solution. Where all the stakeholders originally–every single week–have to come to the site to inspect the progress of the construction; now they’re able to take a HoloLens, instead of pulling up a paper-based blueprint, to really be able to compare where the project is supposed to be vs. the reality onsite and be able to spot a lot of the construction mistakes much more in advance. For example, you are supposed to build this pillar here, but you can’t really see that based on a 2D blueprint on paper. But you can immediately see that from a 3D representation of a visualization though HoloLens, you know that this thing is three feet off. And that gives a lot of efficiency to do construction inspection.
V1 Media: Can you describe a little bit more about the HoloLens technology and how that might become more integrated into future engineering?
Li: So HoloLens in general we call mixed-reality technology. It is essentially a technology that can bring lots of virtual reality, virtual data information and virtual objects into the physical world. And if you think of “Iron Man,” we’re essentially changing the construction field and making construction workers become “Iron Man” working on the field. And be able to access this information in front of their eyes, but not only as a heads-up display but also bring this virtual information in context relevant to the physical world. Imagine that on a construction site or in a connective factory you actually can see the real-time data right next to the IoT sensor, instead of seeing it on a 2D screen in a back operating room with somebody staring at a monitor somewhere.
