What It Really Takes To Scale Agentic AI

First it was generative AI, then AGI captured imaginations. Now, it’s agentic AI that’s keeping the C-Suite up at night, as business leaders look for AI that doesn’t just generate responses, but acts, decides and delivers real business value.

Boardrooms are obsessing over it, investors are betting on it, decision makers are piloting it and Gartner analysts are projecting that by 2028, a third of enterprise software will include agentic AI — up from just 1% in 2024 — powering 15% of daily business decisions to be made autonomously by that time.

But for all the hype, something isn’t clicking and most organizations are still stuck in their pilots, many of which never scale into production or end up failing during deployment. For context, 85% of AI projects fail. And when you ask the people building these tools what’s really going on, the consistent theme is that while they have AI agents, they don’t really have the ecosystem to support them.

Building Infrastructure First

Aishwarya Singh, SVP of Digital Collaboration Services at NTT DATA, has seen that story unfold up close. “The biggest economic bottlenecks include the high initial investment in infrastructure and technology, the cost of integrating AI with existing systems and the need for specialized talent to manage and maintain AI systems,” she told me in an interview.

In theory, agentic AI should reduce cost and complexity. But in practice, it adds a new layer of both — especially if companies treat it like a product and not a process. “Many leaders underestimate the time, effort and resources required for successful integration,” Singh said. “Ignoring this can lead to project delays, cost overruns and suboptimal performance.”

Launched in March of this year, NTT DATA’s new Agentic AI Services, built with Microsoft’s CoPilot Studio and Azure AI Foundry, aim to fix that — not just by deploying agents, but by supporting the entire lifecycle: advisory, build, implementation, monitoring, retraining and optimization. It’s AI infrastructure as a managed service, and it’s already being deployed internally across the company.

“In our own internal ticketing systems, productivity improved by 50 to 65%,” Singh said. “We build agents across ticket types and link them together across omnichannel LLMs so that we can layer on new automation consistently via voice, email and chat.”

The AI Talent Deficit

But that lack of infrastructure or ecosystem, as industry experts put it, isn’t the only thing holding agentic AI back. Another issue, and perhaps even bigger, is the AI talent deficit. According to a recent Accenture study of 3,400 executives and 2,000 enterprise projects, only 13% of AI initiatives are delivering significant business value. The reason? Companies are spending three times more on technology than on people — and that AI skills gap is showing.

“Talent readiness is one of the biggest barriers to scaling and unlocking value for companies,” said Jack Azagury, group chief executive for consulting at Accenture. “One can invest in all the available Gen AI tools, but if your employees don’t know how or why to use them, the value will simply not be realized.”

Singh agrees, noting that this increasingly wide AI talent gap is why NTT DATA is investing in upskilling 200,000 employees and certifying 15,000 GenAI experts this year alone. “This has also introduced a lot of ideas around how we can leverage this technology to improve our own business performance, which is leading to incredible new innovations,” she said.

The AI Deployment Debacle

When you move past the talent debacle, you face another even greater problem in actually deploying AI. A recent working paper from the National Bureau of Economic Research tracked AI chatbot use across 7,000 workplaces and revealed that these chatbots had almost no significant impact on pay or hours worked in any occupation. Despite wide-scale adoption, the study found that on average, AI only saved employees 3% of their time. Of that, just 3 to 7% was passed on as higher compensation.

Even more striking is the finding that most employees redirected their saved time toward other tasks, often ones created by the AI system itself — editing AI output, rechecking hallucinated facts, or adjusting for tone. In other words, the technology added more complexity than it removed.

That’s similar to what IBM also found in a separate study which showed that only 25% of AI projects deliver their expected ROI. And Informatica’s most recent report reveals that data quality and integration issues remain the top reason most AI projects fail.

The bottom line is that AI agents don’t scale because enterprises don’t yet know or understand how to scale the surrounding conditions.

The Post-Deployment Complexity

If you manage to deploy your AI agents successfully, you now have to worry about what happens after deployment. Even the best AI agent needs a team behind it: developers, data stewards, security architects, trainers, ethicists and more. This is where most companies face the biggest challenge, according to Singh — not in deploying an agent, but in managing what happens next.

“Post-deployment, [agent management] involves regular updates, performance tracking, security audits and alignment with evolving business goals,” she told me. “A significant pain point we are hearing from clients is how to best manage the surge of agentic AI agents within their organizations.” That’s exactly where many organizations are flying blind, building AI agents without a strategy for how to keep them running, governed and optimized at scale.

To address this growing challenge, Singh noted that NTT DATA is starting to introduce guardian agents and Red Teaming agents — models designed to monitor security, compliance, and operational integrity as agents proliferate across functions — into their managed stack.

Where Real ROI Begins

So what’s working? If agentic AI is burdened by all of these complexities, why’s there still such hype about it, so much that many companies around the world plan to have an agentic AI pivot? Singh’s answer is that in spite of the complexities and setbacks, agentic AI has real-world use cases that offer a glimpse of its potential when properly deployed.

“We are seeing top use cases in IT services, tactical process automation, customer service and multiagent models for more complex tasks like inventory management,” Singh explained. “Clients can expect a payback period of 6 to 12 months. Productivity gains often become evident within the first few months.”

But those results only show up when there’s a complete system behind the agent — one that includes change management, talent development, cross-platform integration and ongoing optimization. As Singh noted, companies that succeed are the ones who prototype quickly with tactical use cases, and hyperscaler-aligned teams ready to scale within their existing cloud environments.

Getting The Basics Right

Agentic AI won’t scale because you hired a vendor. It will scale because you built the internal architecture — including technical, organizational and human — to support it. That’s the big message for companies planning to scale agentic AI today, according to analysts, projections and several enterprise case studies.

Every agentic AI success story starts with getting the basics right — data, talent and infrastructure. And that, said Singh, requires a lot of planning. The question isn’t whether companies can scale their agentic AI projects. It’s whether they are ready to do what it takes to get it there.