Why Enterprise AI Still Can’t Deliver On Its Promise

Companies are pouring billions into enterprise AI, but poor data, weak infrastructure and missing strategy keep killing results.

getty

Across boardrooms, enterprise AI has become the biggest line item in the innovation budget — yet it’s also become the biggest source of anxiety. Companies are deploying large models, generative assistants and predictive systems at record pace, but the results for many aren’t keeping up. Behind the PR and pilots, there’s a growing sense across the industry that something’s broken. AI was supposed to unlock new value but for many, it’s only adding layers upon layers of noise.

AI critics like cognitive scientist Gary Marcus and tech columnist Ed Zitron have continued to question where the true moat for AI companies like OpenAI and Anthropic lies, with Zitron popularly describing generative AI as “a financial, ecological and social time bomb” back in February. For Marcus, it’s possible to build truly great AI systems, but just not with current mainstream models or approaches. He argues that right now, LLMs are dishonest, unpredictable and potentially dangerous.

Andrew Frawley, CEO of Data Axle, believes the major problem begins before even a single line of code is written. “The performance gap in enterprise AI isn’t a surprise. This is what happens when ambition outpaces readiness,” he said. “Many companies have invested in AI like it’s a product, not a capability, expecting they could flip a switch to unlock immediate value. But AI doesn’t operate in a vacuum. It’s a high-performance engine and too many are trying to run it on dirty fuel.”

AI Without Direction

Frawley did not mince words when I asked him the big reason for this problem. “The real issue isn’t the technology itself, but the foundation,” he told me. “Companies are obsessing over models while neglecting or under-nurturing the one thing those models rely on: data.” Fragmented records and siloed systems have become default conditions in most enterprises. AI only exposes those fractures faster and at scale.

“Some brands, blinded by AI’s possibilities and potential, rush for immediate deployment while bypassing the crucial, foundational work of establishing a data infrastructure,” he explained. “The most critical steps — which include establishing data ownership, building governance into workflows and enforcing quality standards — often get pushed aside in the interest of speed.”

But that, according to Frawley, always results in misfires that damage trust. “Imagine a brand sends a ‘customer loyalty’ promotion to someone who just filed a major service complaint,” he explained. “The AI, lacking the critical information of a recent negative interaction, makes a confident but tone-deaf mistake.”

His is a view echoed globally. Udo Foerster, CEO of German consultancy Advan Team, sees similar dysfunction among the businesses he advises. “Too often, there’s no clear strategy. No defined goals. Just vague roadmaps and no system for tracking progress,” he said.

“And here’s the other big problem: AI isn’t a plug-and-play miracle. Companies assume their data is ‘good enough’ and skip data inventory and cleanup. They don’t adapt their processes — they just try to bolt AI on top. That’s a recipe for unreliable results, growing mistrust, and failed projects. Ignore the fundamentals, and you’re building on sand.”

The Infrastructure Bottleneck

For all the talk of algorithms, it’s the invisible plumbing beneath AI that’s doing the damage. “Bad data rarely announces itself,” enthused Frawley. “It operates silently, creating a slow erosion of trust and value that companies don’t notice until it’s too late.” That damage isn’t hypothetical, but real, measurable and often expensive.

“Flaws are amplified. This can lead to irrelevant content, misplaced or mistargeted ads that can waste millions of dollars in spend, or an inconsistent narrative that causes brand confusion and alienates critical audiences.”

Foerster put a number on it. “Easily 60 to 70% of AI’s growing pains trace back to data infrastructure,” he told me. “It’s overlooked because it’s invisible — not a flashy innovation; just the hard grind of IT and governance. But without it, no AI system in a company will ever deliver.”

Ken Mahoney, CEO of Mahoney Asset Management, flagged another overlooked bottleneck: The physical limits of AI’s appetite for energy and infrastructure. “Even if you have all the money you want to spend, it may not be possible,” he said. “Transformers, power equipment and cooling equipment all have limited availability at the moment. The energy companies are backlogged for years for equipment, so there is a ton of money ready to be spent, but you cannot create power and equipment, and data centers out of thin air.”

AI’s Context Problem

While many enterprise leaders have been told AI can think and some even outrightly believe it, the reality is that most AI systems still can’t understand context or nuance like humans would — a problem that Marcus has often talked about and one that may not go away anytime soon, despite how much money has been poured into building LLMs.

Frawley says that without clear strategy and clean data, models confidently push the wrong action. “Deploying AI on fragmented or inaccurate data is an act of self-sabotage,” he said. “It will amplify existing flaws, erode the quality of analytics and introduce a false sense of confidence in misinformed decisions.”

Foerster offered a personal example: “At my favorite hotel, I tried booking our go-to room — number 138, overlooking the spa gardens. The AI at the call center didn’t recognize that my wife and I are regular guests or that this is our preferred room. I was pretty frustrated. The issue was fixed only when a human stepped in and turned the AI off.” In AI-driven systems, those small misses scale quickly — and quietly — until customer confidence breaks.

The same risk extends to more complex deployments. “Generative, and especially agentic AI, act as amplifiers,” Frawley said. “With clean, well-structured data, they can accelerate progress. With fragmented or inaccurate data, they amplify errors and bias at speed, autonomously executing actions, pushing a business further in the wrong direction before the problem can be detected.”

What Readiness Really Looks Like

The solution isn’t to abandon AI. It’s to approach it like any other critical enterprise transformation — with strategy, systems and accountability. Frawley pointed to clients in the telecom and healthcare sectors who couldn’t even unify a customer record before attempting advanced personalization or churn prediction.

“We worked with them to create a single customer view by unifying, deduplicating, and enriching their data from each source, whether digital or offline, across all channels,” he said. After the fixes, one client saw a 15% improvement in identifying duplicate records. “These improvements resulted in a reduction of wasted ad spend and more meaningful, strategic customer communications at scale.”

Foerster also shared a similar case from manufacturing. “A German firm had big ambitions for predictive maintenance. But the rollout was stuck in neutral,” he noted. “The breakthrough came only after they standardized machine data, linked it with master data, and built a central data warehouse. Suddenly, the models became accurate. Equipment downtime dropped 30%, and the company gained real value — smoother maintenance, better production flow and a clear boost in productivity.”

Moving Beyond AI’s Promise

If there’s one thing these leaders agree on, it’s this: AI won’t fix a broken business. Rather, it will expose it.

“If I had five minutes with a boardroom, I’d ask one question,” said Frawley. “Would you bet your reputation, your job, and your company’s future on the accuracy of your data? If the answer is anything but an immediate ‘yes,’ the business is not ready for AI.”

Mahoney added a financial lens to the matter: “We want to know if you are a newer startup or an existing company that claims AI will bolster your business, we want to know the step-by-step plan, and how will you quantify how you are growing or creating more efficiencies.”

And Foerster’s final thought was rather blunt: “What specific business problem do you expect AI to solve and how will you measure success? If you can’t answer that, neither the best algorithm nor the biggest budget will help you.”

Until enterprises get clearer on the answers, the promise of AI will remain just that — a promise.