By Andrew Tan
There's a gap between the story being told about AI in the enterprise and what companies are actually experiencing on the ground. You could watch this play out across industries for a while now, and the pattern is consistent enough that it's worth naming directly.
The pitch is familiar: AI tools will automate the repetitive work, amplify your team's output, and ultimately let you do more with less. The reality, for most organizations, looks quite different. The executives I speak with are largely describing the same experience — AI projects that showed early promise in demos and pilots, then ran into friction when exposed to the noise of real production environments.
This isn't an argument against AI adoption. It's an argument for being precise about where AI actually delivers value versus where it adds cost and complexity without a corresponding return.
The deployment failure pattern
The first thing that gets lost in AI coverage is how often production deployments fail quietly.
Announcements of AI initiatives tend to generate press. The quiet rollbacks that follow tend not to. But when you talk to operations teams candidly, the reversal pattern is common — systems that worked in controlled testing, connected to clean data and well-defined inputs, that degraded when exposed to the variability of real customers, real data, and real edge cases.
Customer-facing AI deployments have been particularly prone to this. The tolerance for errors in customer interactions is low, and the compounding effect of getting things wrong repeatedly erodes trust faster than any initial efficiency gain can offset. Teams that replaced human capacity with AI and then had to reverse course found themselves spending months rebuilding, often with more urgency than before.
The lesson isn't that AI customer interaction tools don't work — it's that the failure modes are underestimated during the planning phase, and the cost of a failed rollout exceeds the projected savings even when the initial deployment looked promising.
The accuracy ceiling
Why do production deployments fail at rates that don't match pre-deployment expectations? The answer is largely in how AI capability is measured versus how it needs to perform.
Benchmarks and vendor demos select for conditions where AI performs best. Production environments don't. The gap between benchmark accuracy and real-world accuracy is consistently larger than teams expect, particularly for anything involving ambiguous inputs, unusual edge cases, or tasks requiring contextual judgment.
In software development — which has been the proving ground for AI productivity claims — the productivity story is more nuanced than the marketing suggests. AI tools are genuinely useful for certain well-scoped tasks: generating boilerplate, explaining unfamiliar code, drafting documentation. But the secondary costs of AI-assisted development are underweighted: code review cycles get longer when you can't assume the same level of reliability you'd expect from an experienced engineer, security review becomes more necessary, and debugging AI-introduced errors can consume more time than writing equivalent code from scratch.
The net productivity effect, in practice, is much closer to neutral than the adoption narrative suggests. The teams I've seen extract real value from AI coding tools have been disciplined about scope — using AI in a narrow, well-supervised lane and keeping human judgment in the loop for anything that matters.
There's also a question of whether reliability improves sufficiently with more capable models. The structural challenge is that AI systems are fundamentally probabilistic — they approximate, they extrapolate, and their confidence doesn't reliably track their accuracy. Newer models are better, but the same category of failures persists. The question isn't whether AI will ever be reliable enough, it's whether the current generation is reliable enough for the specific task you're considering, and that requires honest evaluation rather than optimistic extrapolation.
The real cost equation
Even setting aside the reliability question, the economics of AI deployment have shifted in ways that deserve scrutiny.
When AI tools first entered the enterprise, pricing was structured to drive adoption — flat subscriptions that made ROI calculations appear straightforward. Many of those pricing models were, in retrospect, being offered well below the actual cost of providing the service. As the market has matured and providers have moved toward pricing that reflects real operational costs, the economics look quite different from the projections that justified many initial investments.
The teams that made commitments based on early pricing are now navigating a different cost environment. Usage-based pricing models mean that scaling up AI adoption increases costs non-linearly. The math that justified a pilot may not survive contact with production usage volumes.
There's also the indirect cost of integration overhead, maintenance, and the ongoing work of keeping AI systems calibrated as underlying models and APIs change. These costs are consistently underestimated in project planning and rarely appear in the productivity gain calculations that AI vendors highlight.
The honest ROI calculation for AI adoption needs to include the full cost picture: inference at realistic usage levels, integration and maintenance overhead, the cost of failures and rollbacks, and the opportunity cost of the engineering time spent managing AI systems rather than building product.
What this means for data infrastructure
The AI productivity story has a specific texture in this space worth unpacking.
The appeal of AI for data workflows is real: generating transformation logic, scaffolding pipeline boilerplate, navigating unfamiliar APIs. If AI could reliably handle these tasks, the productivity gains would be meaningful. The challenge is that data pipelines have near-zero tolerance for silent errors. A transformation that produces plausible-but-wrong output isn't just a bug — it's a corruption that propagates downstream before anyone notices.
The teams that handle this well use AI as a first-draft accelerator for well-defined, reviewable tasks, with automated validation and human review before anything touches production. That's a meaningfully different model from "AI replaces the engineer" — it's more like a junior colleague who needs supervision. That framing leads to better outcomes than treating AI as a reliable autonomous agent.
What doesn't work is using AI in the parts of data engineering where precision is non-negotiable and errors are hard to detect — schema transformations, data quality rules, anything that feeds downstream analytics that people make decisions with. The productivity gains in that zone tend to be negative once you account for the debugging and remediation work.
Calibrating the expectation
At layline.io, we've watched our customers navigate these trade-offs, and the pattern among teams that do it well is consistent: they're systematic about where AI helps and where it doesn't, they insist on validation at every stage, and they treat AI output the same way they treat any external input — with appropriate skepticism until it's been verified.
The AI productivity gap isn't closing on its own. The teams that navigate it well are the ones being precise about where AI genuinely adds value — and staying disciplined about everything else.
A few questions that have proven useful before any AI deployment in data workflows:
What does a failure look like, and how quickly would we detect it? Silent errors in pipelines are categorically more dangerous than visible failures. If the answer to "how would we detect it?" is "we'd notice when the numbers look off," that's not a detection mechanism.
What's the full cost at production scale? Usage-based pricing means the economics at pilot scale don't predict the economics at full deployment. Model it before you commit.
What's the rollback path? Given how often AI deployments require reversal, any adoption that doesn't include a tested rollback path is taking on more risk than the productivity potential justifies.
The upside of AI in data infrastructure is real. So is the downside of getting it wrong. The teams that capture the upside are the ones who go in with clear eyes about both.
Building data infrastructure where reliability isn't optional? Take a look at layline.io — the Community Edition is free to explore.
Andrew Tan is a serial entrepreneur and founder of layline.io, building enterprise data processing infrastructure that handles both batch and real-time workloads at scale.
