Trust, But Verify The Actions Of Your AI Agents

When it comes to the benefits seen from AI agents so far, it’s real “meat-and-potatoes” stuff: 66% in a recent survey, report increased productivity, 57% say they are seeing costs savings, and 55% say AI agents have sped up their decision making. The more “game-changing” stuff – enhanced innovation and opening up new revenue sources – are still lower on the list, cited by 35% and 29% respectively.

The survey of 300 senior executives, released by PwC last month, finds evidence of these basic benefits, as well as plenty of money flowing toward agents. Almost all, 88%, say their team or business function plans to increase AI-related budgets in the next 12 months to develop and deploy agentic AI. More than one in four, 26%, are boosting such budgets by more than 50%. Seventy-nine percent say AI agents are already being adopted in their companies.

Still, most (68%) report that half or fewer of their employees interact with agents in their everyday work. “Few businesses are connecting agents across workflows and functions, yet that’s where the real value lies,” the PwC researchers stated.

What will it take to deliver effective agentic AI beyond the promises of productivity and cost-savings, which is a hallmark of every technology before it? Experts and leaders across the business landscape point to the need to pay close attention to factors such as trust, employee preparation, data and corporate culture.

“The rapid surge of AI and agentic models will democratize tech like never before,” predicted Elise Houlik, chief privacy officer at Intuit. Already, they are being widely applied in “a myriad of disciplines, including marketing campaign creation, contract reviews, and regulatory compliance.”

However, many organizations may not be ready to embrace these advantages on a large scale. “The readiness of enterprises and their technology teams to integrate such advanced AI solutions varies considerably,” said Dr. Kwamie Dunbar, associate professor of finance at Worcester Polytechnic Institute. “This disparity underscores a recognition of AI’s potential benefits, tempered by a deliberate approach to its implementation.”

Many organizations “are not fully prepared to integrate these advanced technologies,” agreed Leonard Kim, chief product officer at Hyland. Agentic AI implementations need to include the “upskilling of teams to bridge the AI knowledge gap,” said Kim. Along with that, “AI needs to be seen as a tool to enhance human capabilities rather than replace them.”

Integrating agentic AI into existing workflows ”demands substantial changes in organizational processes and culture,” said Dunbar. Add to that a “lack of data readiness. AI systems require consistent, clean, and well-organized data to function effectively.”

Agentic AI – supporting autonomous applications, where it’s value is surfaced – requires a strengthening of “cross-functional alignment between technology, business and compliance teams,” said Prashant Kelker, chief strategy officer with ISG.

Trust in autonomous AI agent is another challenge, as revealed in the PwC survey. Thirty-nine percent of executives still do not trust handing over tasks to agents, and 35% are concerned about maintaining human oversight and accountability.

To gain more trust in unleashing autonomous agents on critical workflows, companies at the forefront of agentic AI face a critical challenge: balancing autonomy with user control, said Ashok Srivastava, chief data officer at Intuit. The key is to “incorporate adaptive transparency, ethical safeguards, and context-aware learning to empower customer decision-making.”

To this end, Kelker advises the establishment of “fail-safe mechanisms” across agent systems. This consists of “designing override systems to regain control in case of undesired agent behavior.” This includes the creation of “simulation environments, as well as bespoke simulators for testing agent behavior in controlled conditions.”

Such trust also needs to be managed “intuitive human-AI collaboration, ensuring efficiency while preserving user authority,” said Srivastava. Without trust and confidence, agentic AI systems’ ability to autonomously plan, reason, and execute tasks will be irrelevant. “Striking this delicate balance will be crucial for the long-term success of AI-driven businesses,” he said.