Future Forward Interview:Â Advancing Construction Monitoring
Mani Golparvar-Fard is assistant professor of Civil Engineering and Computer Science as well as the director of the Real-time and Automated Monitoring and Control (RAAMAC) lab at University of Illinois at Urbana-Champaign. His award-winning work in the area of automated construction performance monitoring using video and imagery inputs and 4D building information models has been strengthened by partnering with construction companies to put these approaches into practice.
I2: Could you briefly describe the work that you do at the lab?
Golparvar-Fard: I have been at the University of Illinois for three years now. When I came, I founded a lab that was originally founded at Virginia Tech. The lab is called RAAMAC for short, which stands for Real-time and Automated Monitoring and Control. In my lab, where I have 12 people now, we specifically focus on answering fundamental research questions on visual data and analytics for monitoring performance of construction projects.
We’re interested in analyzing images and video to see if we can automatically recognize objects and measure performance. Performance could be the physical progress on construction sites, it could be productivity, it could be safety, it could be quality. It’s all based on visual data (images and videos).
We want to remain agnostic on how this visual data is captured. In the lab, we take advantage of images that are captured with time-lapse cameras; we like to leverage images captured by commodity smartphones; and we also like to take advantage of images captured with consumer-grade cameras or more recently with drones.
To perform performance monitoring, we also take advantage of building information models (BIM). We basically tie-in BIM with images and videos to perform our performance monitoring by creating new algorithms.
I2: Are there organizations and foundations that support your research?
Golparvar-Fard: The lab is funded by various organizations. The majority of the funding comes from the National Science Foundation. We have also received funding from various groups, including the National Center for Supercomputing Applications to the American Association of Railroads to FIATECH to Construction Industry Institute. There are a range of these sponsors.
We do basic research, creating prototypes in the lab. Once we have a prototype, we typically partner with construction companies to pilot these solutions to understand the value they add to the construction processes. Once the validation process is conducted, we graduate the prototypes, and they are used by a startup company, or a third-party company can acquire a license to the technology. There are a few companies that have done this in the past.
We are looking at the whole spectrum, going from idea to a validated prototype that then goes beyond the lab to a working solution.
I2: What is the focus of the startup that you’re involved in?
Golparvar-Fard: The most recent startup that I have is Reconstruct. We offer a visual production management system for construction projects. It predicts risk and execution of a project against its schedule and can provide that detail to project stakeholders.
The intelligence is provided by machine learning algorithms that we create. One beautiful thing is that in construction, you always have plans for everything. The plan provides a baseline that we can use to identify changes with respect to that plan. This could be quality change, safety change or progress change.
When it comes to productivity, it’s a slightly different scenario. We’re interested in tracking activities that construction workers and equipment perform on the jobsite. We typically don’t plan for those types of activity in detail, so there’s no real benchmark. It’s mainly us performing what is typically called workface assessment or activity analysis on a jobsite.
I2: Who is the target customer of such productivity and performance forecasts?
Golparvar-Fard: There are a number of problems that we’re working to address for owners and operators.
The first is inadequate communication. There are a number of reports, with the most recent one from McKinsey, that show that understanding among contractors, subcontractors and owners on how much progress is being achieved on the jobsite every day is actually different. If you see reports from each, they don’t match.
There has been a surge of startup companies and solutions for improving communication, but we still don’t communicate performance problems. We wait for a week before going to a contractor coordination meeting to find out that a subcontractor had a problem last week that could have been addressed to keep the project on schedule. As a result, there is flawed performance management on jobsites.
Over the last 10 years, planning has significantly grown. Companies are achieving really good planning when it comes to creating a master schedule, understanding what needs to be done and planning for that two months and even two years down the road. When it comes to short-term planning, planning for subcontractors to execute tasks on a weekly or daily basis, there’s a huge gap in the ability to create reliable work plans.
People typically involved in this planning are not the people onsite. What’s missing is a connection for these planners to know exactly what’s happening on the jobsite. The techniques for construction monitoring have been advancing rapidly, we use Earned Value Analysis, we use Plan Percent Complete and Lean Construction. These are all retroactive metrics where we can only learn about mistakes in the past. It’s useful if the tasks are repetitive, but in construction, things don’t repeat.
Instead, there need to be metrics to proactively identify potential problems before they surface on the jobsite. These problems have been motivating the type of predictive analytics that we’ve been working on to make sure we can bring transparency to where the problems are and where they are likely going to be. We can also bring accountability by showing who was expected to perform a specific task on a given day by bringing traceability. The capability to track the commitments people make in coordination meetings is key.
I2: Do you want to speak about some of your project partners and the technology they are using?
Golparvar-Fard: I’ll speak on two sides, both the lab side as well as my company. At my lab, we have strong partnerships with a very large number of construction companies. The company that has engaged the most is Turner Construction. That’s partly because I used to work for Turner. We work with Turner across the country, in New York and Chicago as well as Northern and Southern California.
Other companies that we work with are Clark Construction, Gilbane, Clayco, Consolidated Contractors Corporation in the Middle East and several companies based in Japan. These are some of the partners that we’ve had over the years.
I2: Is the work both process and project specific?
Golparvar-Fard: We like to pilot at the project level. The connections that we have are at the company level, but the pilots are all at the project level.
Most important in all the pilots is assessing the value that our solutions add and mapping our solutions into their existing workflows. Creating a technical solution is great; that’s what everybody does. There’s a whole lot of effort to take a prototype and provide something that can actually be used on a daily and weekly basis. We work with our partners to establish those workflows, and it’s been very important to us to have access to project work.
I2: You mentioned that the inputs are increasing. Can you speak about the information cloud created around every project?
Golparvar-Fard: The capability of taking pictures has significantly grown over the last five or 10 years. People have always been capturing photos on construction sites, and time-lapse cameras have been around since the 1980s. Now every subcontractor has a point-and-shoot camera. If they’re not using a camera, they are using an app (Latista, Plangrid, Fieldwire) that lets them take and annotate pictures.
In addition to all the pictures being captured, companies hire professional photographers to take pictures, and there are companies that monitor with time-lapse cameras (Multivista, EarthCam, JobsiteVisitor, OxBlue). Laser scanning is also being done on a massive scale. Companies are scanning every concrete placement before and after each placement.
Drone use is also on the rise, as 40 percent of the companies that currently have FAA 333 exemptions are using drones on construction sites. Sixty percent of pending applications are also in construction. So obviously there’s a huge opportunity. The problem is they are simply increasing the volume of photography that is being captured.
There’s a huge amount of visual data that gets captured, mostly for liability issues.
We are tapping into the existing visual data to try and bring intelligence to it and map the images into project control workflows. People need to link planning to reality capture from the drone to bring intelligence to it and offer metrics that are meaningful for the execution of the project. That’s the sweet spot that we’ve tried to address.
I2: Are there photogrammetric processes to make models from images in your solutions?
Golparvar-Fard: I teach a class on visual sensing where I spend about five sessions discussing the algorithms and science behind photogrammetry-based solutions. We have a few intellectual properties on this as well. We think we may have something that is faster and better, but it’s obviously a quickly growing market where there is a lot of competition in construction and beyond.
Microsoft has it’s own photogrammetry solution, as does Google and Apple. Being able to automatically create 3D models from pictures, or visual data in general, is a rapidly growing market. However, it’s just one piece of the puzzle.
We need to be able to bring this into project control workflows, which requires integration with BIM, schedules, billing systems, cost estimation and various common jobsite workflows.
I2: Is the focus primarily on new construction?
Golparvar-Fard: No, we also do building diagnostics. In terms of existing residential buildings, a large stock of them require renovation over the next 10 to 30 years. We’re dealing with a problem of not being able to measure the actual thermal performance of a building envelope to understand what steps need to be taken to improve energy efficiency.
We’ve taken the basic concept of thermal photography, which building auditors use. We’ve transformed it into 3D models that show the thermal distribution in space. We measure R-value, which is thermal resistance of the building envelope. Then we have algorithms to calculate the cost of the measured R-value. We can actually present energy loss costs to homeowners.
If you’re a homeowner, you can go to Home Depot and rent one of these cameras. You can scan your facility, upload the photos to our web-based system, and we give you a report. The reports can then be provided to contractors to give you a quote on energy renovations, including details on the available tax incentives to homeowners that address their energy inefficiency.
It’s a great opportunity to understand objectively the cost associated with energy loss and the cost justification to invest in retrofits.
I2: I find it unique that you’re not only cost-justifying such things as retrofits, but in your prototypes you’re also cost justifying your solution to the overall construction process. You obviously place a great importance on value.
Golparvar-Fard: I have been growing a philosophy: In academia, the expectation is that we contribute to science. Most research stops at a prototype, and many don’t take their research that far. Industry expects us to have a working solution. The gap between a prototype and a working product for construction monitoring is what we address. There have been so many startup companies, but very little work to formalize and educate and integrate this into university curriculum. My personal interest is to make sure to bring entrepreneurial thinking to the classes that I teach and to my own research. More importantly, it’s a validation of my own work if I’m creating something that has application and makes a difference in construction.
About Todd Danielson
Todd Danielson has been in trade technology media for more than 20 years, now the editorial director for V1 Media and all of its publications: Informed Infrastructure, Earth Imaging Journal, Sensors & Systems, Asian Surveying & Mapping, and the video news portal GeoSpatial Stream.
