Physical AI is Moving Things

Rendition of Unmanned Aerial Vehicles With Self-Organizing Capabilities at Sunset

getty

The Ukraine war, now into its 4th year, is an example of how physical autonomy of low-cost, AI powered weapons like drones can minimize loss of human life while invading adversaries in their own backyard. Today, such platforms are remotely managed by a human operator or follow a well-defined and structured leader-follower protocol. The eventual vision is that large groups of drones can engage in swarm-like behavior, similar to bees and ants, “where a distributed, intelligent system that dynamically adapts to changing circumstances in real time, without reliance on a central controller, a designated leader drone, or even an internet connection can overwhelm entire systems and strategies that have not anticipated coordinated and adaptive attack, self-healing networks, or swarm intelligence”.

A previous article covered the races (autonomous and human) at the famed Laguna Seca Raceway in which autonomous race cars performed at the race-track, just prior to the INDYCAR race 3 days later. Apart from racing, it discussed the role of cooperative behavior in complex multi-agent players in physical AI applications like Waymo’s L4 (no human driver in specific areas of operation) autonomous driving initiatives (Figure 1).

The State of Physical AI Today

Digital AI involves sharing of data to deliver applications like LLM (Large Language Models), LRM (Large Reasoning Models), search, voice/image recognition, video analytics, marketing and software coding. It is already producing gains in white-collar productivity, focused market targeting and knowledge access. Progress has occurred primarily with commercial investments and innovation (although search started off with early research work at Stanford and Berkeley).

Physical AI uses data and real-time connectivity to move atoms and Things. It is an exciting and emerging field with significant applications in uncontrolled, semi-controlled and controlled environments on land, sea, air and space. The focus is on productivity, safety, consistency and 24×7 operation of capital assets.

The most public application is the AV (Autonomous Vehicle) revolution which solves a pain point for consumers who prefer not to drive, and addresses the shortage of drivers and time restrictions in trucking and industrial applications. Unlike digital AI, error tolerance is dramatically lower, and safety is paramount. AI training brittleness can lead to devastating consequences. Currently, the entities driving this are massively funded commercial entities like Tesla, Waymo, Baidu, Didi and automotive OEMs. Uber announced a collaboration with Lucid (U.S. based luxury EV company) and Nuro (last mile autonomy for delivery) to launch driverless ride-sharing services. Volkswagen also announced the launch of its autonomous shuttle service in Europe and USA. Waymo struck a partnership with Toyota to integrate its autonomy suite into Toyota passenger cars for personal autonomy. Trucking autonomy in North America is dominated by Aurora, Waabi, Gatik and Daimler (all are in trials currently). Amazon. Caterpillar, John Deere, Komatsu, Pronto and Oshkosh are already automating the movement of construction, mining and agricultural equipment or goods in warehouses. Players in the shipping, undersea, drone and outer-space arenas are embracing the use of physical AI to enable autonomy in these environments.

Other notable happenings in the Physical AI space include:

1. In the midst of U.S. sanctions and a trade war, the Chinese government is investing heavily in creating an invention ecosystem around physical AI. It aims to lead in the AI field by 2030. More than 800 companies are set to participate in this year’s AI forum.
2. The war in Ukraine is creating high pressure innovation in intelligent drones and swarms. The idea is to deploy drones, learn and improve even if it means sacrificing some of them to learn and improve.
3. The U.S. government is paying close attention to the progress of physical AI and drones in the Ukraine. Policies are being implemented that strengthen the supply chain for smaller drones and allow local commanders to use these in training battles to learn and improve (allow use of disposable assets in training).
4. The Trump administration just released its AI action plan for the United States. As opposed to the previous administration, it is much more muscular and aggressive, aimed at unabashedly securing U.S. leadership in this area. This is not surprising since the plan is authored by Silicon Valley venture capitalists Michael Kratsios and David Sacks. Key investments areas include data centers, electrification and semiconductor manufacturing in the U.S., building the next- generation workforce skilled in AI, implementing cybersecurity controls and investing in AI-enabled science through agencies like the NSF, DOE and NIST.
5. The U.S. Department of Transportation just released a white paper that discusses relaxed regulations for BVLOS (Beyond Visual Line of Sight) drones that include fixed wing and VTOL (Vertical Take-off and Landing) for up to 400 ft. altitudes and total maximum weight of ~1300 lbs. This is meant to promote applications like goods delivery and mapping, but it also encourages domestic production of drones for commercial and military applications.
6. In a related development, the Open Compute Project recently announced an initiative aimed at using free-space optics to route data between nodes to meet the demands of high bandwidth, low latency, and energy efficiency in data-intensive applications like artificial intelligence. Led by volunteers from iPronics and Lumentum, it has participation from a wide variety of companies in optics and AI like Coherent, Google, Lumotive, Microsoft, nEye, NVIDIA, Oriole Networks, and POLATIS.
7. As physical AI starts playing a bigger role in defense, players like Palantir and Anduril continue to disrupt the development models of traditional defense contractors. This is identical to the disruption in the AV space where fast moving innovators like Google created a new generation of capabilities in record time as traditional automotive OEMs struggled for direction, talent and innovation.

In the midst of these dynamics, how should university, industry and government organize and invest to ensure leadership in this multi-trillion-dollar arena?

Physical AI SUMMIT and Panel Discussions at Laguna Seca, Monterey

To answer the above question, IAC organized a summit with panel discussions on understanding the current state of Physical AI and how academia, business and government entities should plan, organize and fund this exciting field to achieve scalability and leadership. It occurred during the morning, of the Laguna Seca AI race-car event. Conducted with participation by recognized experts and moderators, the panel discussed global issues and trends in the world of physical AI. It was attended by ~100 representatives from academia, business and government (Figure 2):

The main themes and learnings from the Summit are summarized below:

1. Education and Workforce Development: Academia has a big role to play in training the workforce for this emerging technology. IAC competitions are ideal for growing a rich, experience-based talent base in Physical AI.
2. Government Initiatives and Funding: As discussed above, the Chinese government is investing heavily in innovation in this area. As the Ukraine war progresses, the U.S. government is understanding the power of rapid prototyping and experimental focus to improve and innovate on physical AI capabilities. One of the points discussed by the panel is the need for governments to fund large and ambitious efforts with specific focus areas rather than spreading budgets towards many small projects. The latter approach has been tried in Europe with very little to show for it.
3. Developing dual-use defense capabilities is also important to encourage investment and create profitable business opportunities. The IAC activity (which has DARPA sponsorship) is good example of government investment focus on dual use capability. IAC also represents a bottoms-up approach to progressing physical AI, which is critical to ensure democratization of the technology as it moves forward.
4. Infrastructure Capabilities: Physical AI will require a power grid capable of supporting data centers and machine learning, ideally in a distributed fashion. Government entities need to understand this strategic need and invest in these capabilities. Sustainable and safe nuclear power is an avenue to consider.
5. Semiconductor Capabilities: For physical AI to operate quickly and sustainably with low latency and power requirements, industry investments/consortia in edge computing semiconductor and data center technologies are critical. The central challenge for physical AI is to deliver control decisions using an optimal mix of hardware, software and power.
6. Physical AI Implementation Models: lowering the costs of developing AI models is important to ensure democratization and innovation. The focus should be on breaking up large AI problems into smaller ones that are more focused, use less power and data, and are scalable. Translating learnings from one application to another is important as is the ability of solutions to work across different geographies and weather. One example discussed from the digital AI experience is the transition from foundational models like LLMs (Large Language Models) which “have” small amounts of intelligence across many topics to SLMs (Small or Specialized Language Models) that have deep knowledge about specific areas. This idea is further explored in a recent article that discusses limitations of LLMs and LRMs (Large Reasoning Models) with increasing problem complexity. The implications for physical AI are similar – large end-to-end AI models are difficult to implement, understand and verify. Combining neural networks that focus on specific tasks like perception or localization with neuro-symbolic AI (which is based on human reasoning) is a promising path forward.
7. Open Source Models and Data: given the cost and investments in developing models and acquiring millions of hours of training data, there was a fair amount of discussion on the advantages of open source platforms. While it is true that this can accelerate physical AI applications and provide geographical scalability, it is not clear why companies that have already invested $Bs (like Waymo) would do this and lose their competitive advantage.
8. Outer space already utilizes complex human-knowledge based decision trees that guide actions as long as events are following planned sequences and actions. In case of surprises, it reverts into a “safe mode” and awaits human interaction. In the future, data-based machine learning will be used to guide unplanned actions and/or loss of communication with Earth. Power is critical for learning (especially in-situ), as are efficient (less power hungry and low latency) edge computing solutions for AI engines.
9. Multi-Agent Coordination: Military experience shows that it is easier to automate systems that require minimal human-machine interaction. Complexities arise in cases where humans are involved due to confusion and lack of training. Solving the multi-agent coordination problem between robots and humans is a critical gap today in wider deployment of physical AI. One example is a situation where autonomous cars interact with other autonomous and human driven cars and pedestrians on public roadways (Figure 1). At higher speeds, this becomes even more challenging due to reduced latency decision-making and vehicle control requirements.
10. Solving the AV problem is critical for reducing traffic fatalities. The low speed AV (< 35 mph) problem is already solved (Waymo, et al.). As speeds increase (highway speeds to 90 mph), safe operation of AVs becomes challenging. Low latency perception, compute and decision making, and accurate digital twin models for safe vehicle control needs focus from academia and business. The IAC competitions are certainly great tools for nurturing this, and enabling a bottoms-up transfer of IP to commercial applications. Similar factors exist in defense technology where the need for autonomy in low cost drones and other such platforms is important to minimize loss of human life, and can be translated to commercial applications.
11. Paul Mitchell, the CEO of IAC and one of the moderators at the Monterey Physical AI Summit feels that in the space of < 1 year since the last panel was conducted in Indianapolis, understanding and application of physical AI has grown dramatically. The objectives of these panels is to promote networking and progress in physical AI, and this was achieved by the Monterey Summit.
12. Sergio Savaresi, the leader of the winning team at the Laguna Seca, Monterey competition and a panelist at the Summit felt that the model of aerospace players (airplanes and outer space) in using AI judiciously while balancing safety, transparency and efficiency was important. Other physical AI applications should follow this principle.

As the title suggests, physical AI is moving, a lot of things in a lot of spaces. The challenge is to harness it judiciously and safely, and as one of the panelists described, “move fast responsibly”.

I think physical AI is fascinating, and my company, Patience Consulting offers thoughtful analysis and technical/strategic advice in this emerging area.