Pronto, Komatsu & Caterpillar Advance Construction Autonomy

An Autonomous Komatsu HD605-10 powered by Pronto preparing to be loaded by a Komatsu WA700-8 at Komatsu’s Arizona Proving Grounds.

Pronto

Pronto and Komatsu announced a collaboration to enable quarry operators to retrofit existing Komatsu vehicles or purchase new trucks equipped with Pronto’s self-driving system, enabling 24/7 operation with minimal human intervention. This builds on the move to automate blue collar jobs by Caterpillar, John Deere and Oshkosh to reduce safety incidents and address the shortage of human labor in quarrying, mining, agriculture, airport logistics and garbage hauling.

Pronto (“Autonomous Haulage Simplified”)

San Francisco based Pronto.ai was founded 7 years ago to leverage learnings from the AV (Autonomous Vehicle) experience to off-road environments like quarrying and mining. Anthony Levandowski, a pioneer in the AV revolution (ex-Waymo, Otto and Uber Freight), is the CEO. His pivot away from AVs to off-road autonomy is driven by the more immediate business prospects in this sector, as well as significantly lower capital, R&D investments and regulatory requirements.

Pronto’s autonomy stack solutions are platform-agnostic and composed of hardware (sensors like cameras, GPS, LiDAR and radar) and software. The company originally focused on a sensor-light option (using only cameras and GPS). With the recent acquisition of SafeAI, it can now offer higher sensor trim levels (LiDAR and radar) to address other off-road applications where 3D sensing and 360° situational awareness are critical (dust, fog, weather, lighting, and presence of working personnel and other vehicles).

The hardware stack is primarily used for localization and perception to guide vehicle path planning, speed and velocity control. Machine learning and AI are used to fuse the data into perception and localization maps. The software for each specific site is customized to meet the particular operational and industrial engineering requirements that optimize productivity, safety and environmental requirements of the site (among his many areas of expertise, Mr. Levandowski’s educational experience include industrial engineering and operation research). Key site-specific challenges include building a data-connectivity network, flexibility in operations as customer demand and weather patterns change, and operating in diverse weather conditions.

In November 2024, Pronto announced a collaboration with Heidelberg Materials, the world’s largest producer of construction aggregates, for automating its Texas quarry in concert with Komatsu, North America. It uses a fleet of Komatsu HD605-8 haul trucks integrated with Komatsu’s Smart Quarry platform and Pronto’s AHS (Autonomous Hauling System). The goal is to extend this to other Heidelberg sites in the future (3000 sites in 50 countries).

The recent announcement with Komatsu is intended to build on the Heidelberg collaboration, and extend autonomy solutions to other quarry operators who can either retrofit existing Komatsu vehicles (2005 or later models) or purchase new trucks equipped with Pronto’s autonomy stack, enabling all weather operation with minimal human intervention (Figure 1).

Mr. Levandowski’s vision is to bring autonomy to bear for any vehicle on wheels. “For the U.S. to successfully re-shore manufacturing and secure its supply chains for critical minerals, we have to be able to compete on a global scale. Autonomy becomes a strategic national asset, and making it accessible to quarries of all sizes is a crucial mechanism to boost domestic production of materials needed to build everything from semiconductors to infrastructure. This partnership unleashes American innovation to solve a foundational economic and national security challenge.”

Komatsu (“We build machines to answer the needs of society”)

Komatsu is the Japan-based (in the city of Komatsu) conglomerate with a ~$31B market capitalization, a global workforce of ~65K employees, and engaged in selling agricultural, forestry, underwater dredging, mining and construction equipment. Komatsu North America (headquartered in Milawukee, Wisconsin) has ~8000 employees located in Wisconsin, Texas, Illinois, Michigan and Arizona, and focuses its R&D efforts primarily on quarrying and mining vehicles. Agriculture, forestry, and construction efforts are managed through its Japan division.

Komatsu acquired Joy Global in 2017, enabling the integration of Komatsu’s surface mining equipment with multiple brands of Joy’s surface and underground products. The autonomy stack developed for mining (FrontRunner Autonomous Haulage Vehicle) uses a combination of high precision GPS and IMUs, cameras, LiDAR (Ouster recently announced a LiDAR supply agreement for Komatsu’s suite of autonomous mining equipment) and radar along with vehicular sensors monitoring tire and engine parameters. Sensor information is used with simulation models and machine learning to operate the vehicles safely and efficiently.

The requirements for quarrying (typically in semi-urban areas) are different from mining in remote locations and typically use smaller trucks ranging from 40-150 ton payloads, as compared to mining trucks which are much larger (~700 tons payload capacity). Quarry sites also deploy a lower number of vehicles/site, have very different operational requirements and generally do not run 24 hours/day. As a result, the autonomy solutions are different and need to be more price-competitive.

Michael Gidaspow, is the Chief Digital Officer, Komatsu North America. He indicates that the principle driver for its Smart Quarry platform is to address the very real skilled labor shortage (typical sites currently suffer from a ~25% shortage, and likely to grow). Training and retaining these skills is also challenging. Other drivers include safety and operational efficiency. Figure 2 shows the various components of the Pronto autonomy hauling system (AHS) hardware stack mounted on a Komatsu HD605 haul truck.

The sensor stack (not shown in Figure 2) consists of front and rear mounted cameras and a GPS receiver, essentially the basic trim option. Key aspects of the Pronto collaboration include workforce training to ensure human-machine-autonomy collaboration and a rapid translation of customer requirements into deployable solutions in a matter of weeks.

Per Mr. Gidaspow, “Operational efficiencies and lower carbon footprints are clear asks by our customers who rely on Komatsu equipment today. Our partnership with Pronto is a deliberate step to provide the tools our North American customers need to meet this moment. By combining our proven, high performing equipment with Pronto’s cutting-edge autonomous technology, we are strengthening our quarry & aggregate customers so they can productively deliver the materials essential for society’s building infrastructure needs.”

Caterpillar (“We help our customers build a better, more sustainable world”)

With 2024 sales and revenues of $64.8 billion, Caterpillar Inc. is the world’s leading manufacturer of construction and mining equipment, off-highway diesel and natural gas engines, industrial gas turbines and diesel-electric locomotives. Construction was recently deemed as one of the fastest growing industries due to its technology and AI advancements. An earlier article discussed Caterpillar’s autonomy efforts in the quarry sector, the primary goal being to address the acute shortage of trained personnel in this sector, and mining and construction in general. In a nod to this trend, Caterpillar hosted a Construction & Technology Days event on June 3-5, 2025 which gives an inside look into the latest machine technology, customer success stories and digital solutions shaping the modern jobsite, emphasizing safety and efficiency.

Tony Fassino is Caterpillar’s Construction Industries Group President. He discussed the primary drivers for Caterpillar’s thrust into autonomy (remotely controlled or/fully autonomous) – increased productivity, safety and the shortage of a skilled workforce to accomplish these metrics. An interesting example he cited was hauling of construction residual material from ocean barges without putting any boots on the barge. Wheel loaders are transported to the site, cranes are used to place them on the barge, these are remotely controlled to clear the material and then transported to the next location for another barge clean-up (Figure 3).

A recent activity that Caterpillar equipment participated in at the New Jersey shore is shown in Figure 3. Although not automated, it could be. Sand dredged from approved areas located offshore in the Atlantic Ocean is pumped onto the beach through a network of pipes that run to shore, and graded into an engineered template, which is designed to reduce damages from coastal storm events. The surgical precision with which the on-shore pipes are placed to transport the dredged sand to replenish the beach area is amazing. It is conceivable that in the future, this entire operation could be semi-autonomous, with minimal or remote human supervision.

Sensor technology, especially LiDAR is a crucial element of solving these autonomy challenges. According to Andy Vitale, Caterpillar’s Director of Engineering, Autonomy Platform, “Caterpillar is working on both short-range and long-range LiDAR, for use in both mining and construction machines. Mining applications typically require higher speed and thus long-range LiDAR that can detect objects at greater distances. In construction, we are seeing increasing pull for automation features that require mapping of the terrain, which short-range LiDAR is best suited for”. The company recently announced a collaboration with Luminar, a provider of long range LiDAR systems to support Caterpillar’s next generation autonomous solutions.

Physical AI and the Autonomy of Things (AoT™) revolution is progressing in areas that solve critical problems in harsh environments where labor is scarce, capital is expensive, and 24×7 operation is imperative. Advances in sensing, perception, localization, and low-latency computing are making this revolution possible.