All insights
Strategic Brief·9 min read

Before You Select the Tool, Answer These Three Questions about your data

AI tools fail in retail operations most predictably when the data they consume reflects the process as it was designed, not as it actually runs. Three questions asked before tool selection surface that gap every time.

By Jessica Caresse White·
Two diverging process-flow grids overlaid on a floor plan schematic, splitting at three survey crosshair junctions.

Quick answer

Most retail AI pilots fail because the tool is fed a version of the operation that does not exist. The data reflects how the process was designed, not how it runs today. Three questions expose that gap before selection: Where does this data actually come from? How often is it wrong, and how would we know? Who is accountable when it is? The third question almost always reveals that nobody is.

TL;DR

The research on AI implementation failure converges on one finding: the technology is rarely where things break. Here are the numbers that frame this week's argument.

  • Gartner's 2024 research found that 49% of AI proofs of concept never advance beyond the pilot stage

    MIT Project NANDA's 2025 State of AI in Business study analyzed enterprise generative AI deployments and found that only 5% reached production at scale with documented P&L impact. The failure was organizational and data-related, not technological.

  • 60% of AI projects lacking AI-ready data will be abandoned through 2026

    Gartner's prediction, published in early 2025, is already materializing: 42% of U.S. companies abandoned most of their AI initiatives in 2025, up from 17% the year before, according to S&P Global Market Intelligence.

  • Only 7% of organizations say their data is completely ready for AI (Cloudera and Harvard Business Review Analytic Services, 2026)

    A Cloudera and Harvard Business Review Analytic Services survey of 1,574 enterprise IT leaders, published March 2026, found that the gap between AI ambition and data readiness is widespread across organizations of every size and sector, and that most treat it as an exception rather than the standard condition it is.

  • 58% of retailers and direct-to-consumer brands operate with less than 80% inventory accuracy (Fluent Commerce, 2025)

    A 2025 survey of 300 U.S. retail leaders by Priority Software found that the majority of retail operations are still running on data infrastructure that was never designed to feed an AI system.

  • 47% of organizations name unclear ownership and accountability as the primary blocker to scaling governance (Secoda, 2025)

    Secoda's 2025 data governance poll found that unclear ownership and accountability was the leading structural barrier, ahead of budget, tooling, and skills. The governance gap is a people and accountability problem before it is a technology problem.

  • Fixing data quality issues at the source costs ten times less than addressing them downstream

    Research cited across multiple data quality analyses confirms this ratio. The organizations that spend weeks on data readiness before the pilot run faster and cheaper deployments than those that discover the gap mid-project.

What the vendor demo does not show you

You are forty minutes into a demo. The dashboard is clean. The visualizations are exactly what your SVP has been asking for since Q2. The sales lead asks a reasonable question: is your data in a central repository or distributed across systems? You say mostly centralized. You say it because the accurate answer, it depends which team you ask and what day of the week it is, would take twenty minutes to explain and you do not have twenty minutes. You schedule the pilot. This is the moment most retail AI implementations are decided, and it is the moment where the question that determines success or failure goes unasked. The vendor's tool probably works fine. What it cannot do is function reliably on data that describes a process nobody actually runs anymore. The demo environment was built on clean, structured, historically consistent data. Your environment was built on eleven years of workarounds, team handoffs, and systems that were integrated by whoever was available at the time. I have spent enough time in client walkthroughs to know that the number typed into a cell in an Excel summary report is frequently a number nobody in the room can fully trace. The person presenting the summary owns the narrative. The people who built the underlying tabs own the math. Neither group has documented the path between them. When we walk through the process step by step, the manual touch points and copy-paste handoffs become visible in a way that would stop any honest AI vendor in their tracks. This is common practice across the retail organizations I have worked with over the years. A process that grows faster than the documentation around it will produce exactly this condition, regardless of the intelligence or effort of the people running it.

Why retail data is structurally set up to mislead an AI tool

Retail data was built for reporting. Most systems, the ERP, the WMS, the POS, the demand planning tool, were designed to produce accurate end-of-period summaries, not to feed a model that needs to understand the operation in real time. What that produces is a specific kind of data problem that is easy to miss in a selection process: the data is technically present, technically accessible, and wrong in the ways that matter most to an AI use case. Consider inventory. Retailers industry-wide report average inventory accuracy of only 70%, according to iVend's 2025 retail inventory management research. Fifty-eight percent of retailers and direct-to-consumer brands have less than 80% inventory accuracy, based on Fluent Commerce's 2025 inventory data research. And 63% of non-grocery retailers report that siloed data prevents them from responding to real-time demand changes, according to PYMNTS Intelligence's 2024 research conducted with Carat from Fiserv. An AI demand forecasting tool fed 70% accurate inventory data (iVend, 2025) does not produce 70% accurate recommendations. It produces recommendations that are confidently wrong in ways that are hard to detect because the confidence is built in. The model does not know what it does not know. Data silos are the structural cause. Eighty percent of organizations identify data silos as the single biggest barrier to automation and AI, according to Cin7's 2025 State of Inventory Intelligence Report. Inventory teams spend sixteen hours every week, 104 workdays a year, manually syncing data across disconnected systems at a cost of roughly $21,632 per entry-level employee annually (Cin7, 2025). That sixteen hours is evidence that the systems cannot talk to each other, and that the data flowing between them is being shaped by human judgment at every handoff.

  • The process-as-designed versus process-as-run gap

    Every organization has two versions of its processes: the documented version and the version people actually use. AI tools are almost always trained, configured, or piloted against the documented version. The gap between the two is where the pilot fails.

  • Stale data looks current

    Retail data infrastructure rarely flags its own age. A model fed overnight inventory data generates recommendations for a reality that existed twelve hours ago. The recommendation looks precise. The underlying condition has already changed.

  • Teams define the same metrics differently

    Sales defines conversion one way. Marketing defines it another. The AI tool is asked to optimize for a metric that means two different things to the two teams whose data it is consuming. The failure lives in governance: who owns the definition, who arbitrates when the definitions conflict, and what the tool is actually told to do when neither team has resolved it. That governance gap surfaces as a data quality problem, which is why it gets diagnosed late and fixed expensively.

Question one: where does this data actually come from

The question sounds simple. The answer almost never is. In every process walkthrough I have done with a reporting team, there is a moment where the presenter describes a number and cannot fully account for its origin. The summary owner knows the total. The analysts who built the underlying tabs know their sections. Nobody owns the reconciliation logic that connects them. The number exists. Its provenance does not. This matters for AI selection because the tool you are evaluating will be fed data from the same upstream sources. If you do not know where the data comes from, you do not know what the tool is actually learning from. The question to ask, in the selection process, is not: do you have data for this use case? The question is: trace the data backward from the field the AI tool will consume. How many systems does it pass through? How many people touch it between origin and destination? Where are the manual steps, the copy-paste handoffs, the judgment calls that somebody makes when the numbers do not reconcile? Only 34% of organizations know where all their data is stored, and only 39% can fully classify their data, according to Thales' 2026 Data Threat Report. If you are in retail operations and you have not done this trace for the specific data your AI pilot will use, you are in the majority. You are also not ready for the pilot. The practical output of this question is a data lineage map, even a rough one drawn in a working session. It does not have to be a formal data catalog. It has to be specific enough that someone can point to the place where the data last touched a human hand and answer: what did that person decide at that moment?

Question two: how often is this data wrong, and how would you know

Data quality is the most cited cause of AI failure. Gartner's research finds that 85% of AI project failures trace back to poor data quality, and that only 12% of organizations have data of sufficient quality to support AI applications (Gartner, 2025). Gartner also estimates that poor data quality costs organizations an average of $12.9 million per year. But the more useful question is not whether data quality is a problem in the abstract. It is whether the organization has any mechanism to detect when specific data is wrong. Sixty percent of companies do not track the cost of bad data at all, according to research compiled in Precisely's 2025 Data Integrity Trends Report. Data professionals report spending roughly 40% of their time evaluating or checking data quality, according to survey data cited in multiple 2025 industry analyses. Organizations experience approximately 61 data-related incidents per month on average, with each incident taking about 13 hours to identify and resolve, according to TDWI survey findings. This is the operational reality the AI tool is entering. The question for your selection process is not: is our data clean? It is: do we have a detection mechanism that would tell us when the data the tool is consuming has drifted from reality? In retail operations, this drift is predictable. Promotional periods change purchasing behavior in ways that historical models do not anticipate. A supply disruption changes what is actually on hand versus what the system says is on hand. A team change means the person who knew which numbers to adjust and when has left, and nobody has documented the adjustment logic. The AI tool will not flag this drift. It will generate recommendations based on the data it has. The recommendations will be confident. They will be wrong. And they will be wrong in the specific direction that the drift pushed the data, so the errors will compound rather than cancel. Before selecting a tool, the operations leader needs to answer: in the last ninety days, how many times did we discover that a number we were acting on was wrong? How did we find out? If the answer is we found out when something downstream failed, the detection mechanism is not adequate to support an AI implementation.

Question three: who is accountable when the data is wrong

This is the question that ends conversations. Seventy-seven percent of organizations rate their data quality as average or worse, according to Precisely's 2025 research. A substantial share of organizations report struggling with data ownership definition in their governance programs, according to data governance surveys compiled across 2025 research. Only 23% of organizations maintain a formal data governance process with defined roles, responsibilities, and accountability structures, according to Redgate Software's 2026 governance report, even though 77% are actively investing in AI initiatives. The accountability gap takes a specific form in retail operations. A department is named as responsible for a dataset. But within that department, no individual has been assigned accountability for resolving a quality issue, approving use, or answering for the downstream consequences when the data is wrong. The governance structure exists on paper. The person who gets called at 7pm when the inventory count does not match the replenishment order does not exist. Secoda's 2025 data governance poll found that 47% of organizations cite unclear ownership and accountability as the primary blocker to scaling governance. This is not a technology problem. No tool resolves it. It is a decision about who is responsible, made explicit and recorded, before the AI tool is selected. The reason this question ends conversations is that it asks the organization to do something it has been avoiding: assign a name to accountability for a function that has historically been collective, diffuse, and therefore no one's problem. That assignment is uncomfortable. It surfaces existing team tensions. It requires a decision from the C-suite that the C-suite may not be ready to make. This is precisely why it has to be asked before the selection decision, not after the pilot fails. When a pilot fails because the data was wrong, and nobody owns the data, the organization defaults to blaming the tool. The vendor gets replaced. The same data goes into the next vendor's system. The same failure follows.

The contrarian point: a dirty data environment is diagnostic, not disqualifying

The conventional response to discovering that your data is fragmented, inconsistently defined, and unowned is to pause the AI initiative until data quality improves. This is often the right call. But it is worth naming what gets missed in that pause. The process of answering these three questions, rigorously, in a room with the people who actually touch the data, is itself an AI readiness activity. It surfaces the gaps in process documentation, the handoffs that have never been written down, and the accountability voids that have been generating quiet operational losses for years without anyone naming them. I have seen organizations spend four months on data quality work before starting a demand forecasting AI pilot. The pilot deployed in twelve months and produced measurable accuracy improvement because the foundation was real, based on a retail implementation pattern documented in 2026 analysis. The organizations that skipped the readiness work and went straight to the pilot are still cycling through vendors. The three questions do not delay the AI implementation. They determine whether the implementation has a foundation to stand on. A tool selected before these questions are answered is a tool selected on the basis of a demo, not a diagnosis. And a diagnosis is precisely what a lean retail operations team, with limited budget and no room for a failed pilot, needs before it commits.

What could go wrong

  • The readiness assessment becomes a delay mechanism

    Some organizations use the data readiness conversation as an excuse to postpone a decision that is uncomfortable for other reasons. The three questions have concrete answers. If the team has been in the readiness conversation for more than six weeks without producing a data lineage map, a quality baseline, and a named accountability assignment, the conversation has become avoidance.

  • The accountability assignment lands on the wrong person

    Naming an individual as accountable for data quality in a system they do not control produces a governance structure that is worse than no structure at all. The person named carries the blame without the authority to fix the problem. The accountability assignment has to come with the authority to enforce quality standards upstream.

  • The pilot proceeds on a cleaned subset that does not represent live operations

    It is tempting to run the pilot on the best available data, the cleanest subset, the most recent and reliable feeds. The pilot succeeds. The production deployment fails. The gap between the pilot data environment and the live data environment is the gap the three questions are designed to expose.

  • The C-suite treats the readiness findings as a project management problem

    If the answer to Question Three is that nobody owns the data, that answer requires a decision from leadership, not a project plan. Organizations that route the accountability gap through a working group without escalating it to the C-suite are creating a governance structure that has no enforcement mechanism.

  • The governance ownership question gets answered on paper but not in practice

    Thirty-one percent of organizations are still in the early stages of AI governance policy development, and AI governance ranked last when data leaders prioritized their governance concerns, according to the 2025 State of Enterprise Data Governance Report. A named owner who has no capacity, authority, or incentive to act on the accountability is not an answer to Question Three.

The J.Caresse point of view

The selection decision for an AI tool in retail operations is almost always made in a room where the wrong question is being asked. The question being asked is: does this tool do what we need? The question that determines success is: does our data accurately describe the operation this tool will be asked to improve? Those two questions are rarely asked in the same meeting. The vendor has strong incentives to keep the conversation on the tool. The operations team, running lean and already behind, does not have time to run a data audit before the demo ends and the budget decision is due. So the pilot gets scheduled on the basis of a reasonable-sounding answer to a reasonable-sounding question. The process-as-designed gets handed to the tool. The process-as-run, the one with the manual reconciliations, the copy-paste handoffs, the numbers nobody can fully trace, stays invisible. And the gap between those two versions of the process is exactly where the pilot fails. The three questions in this brief function as a forcing function. They make the gap visible before the commitment is made. For a VP running a lean team with one shot at a pilot budget, that visibility separates a successful implementation from an expensive proof that the data was not ready. I have seen both outcomes. The organizations that ask the questions first run slower pre-work and faster deployments. The ones that skip to the tool run fast pre-work and slow, painful post-mortems. The readiness work happens either way. The question is whether it happens before the money is spent or after.

Key takeaways

The research is unambiguous. Data readiness is the leading cause of AI pilot failure in retail operations, and the gap between AI ambition and data foundation is structural, not incidental. Here is what to carry out of this brief.

  • Ask Question One before the demo ends

    Trace the data the tool will consume back to its source. How many systems does it pass through? How many human decisions shape it in transit? A rough lineage map built in a working session is more valuable than a formal data catalog that arrives three months after the selection decision.

  • Build a quality detection baseline, not a quality score

    The relevant question is not whether the data is clean in aggregate. It is whether the organization has a mechanism that flags when specific data the AI tool is using has drifted from operational reality. If the detection mechanism is downstream failure, the pilot is running without a safety mechanism.

  • Name a person, not a department, as accountable for data quality in the pilot scope

    Organizations that struggle with data ownership definition in governance programs are not failing because they lack a named department. They are failing because no individual has accountability for resolving a specific quality issue in a specific dataset. The governance fix is a name, not a structure.

  • The readiness findings require a C-suite decision, not a project plan

    If Question Three reveals that nobody owns the data, the answer is a leadership decision about who does, made with the authority to enforce quality standards upstream. A working group without escalation authority does not resolve an accountability void.

  • The pilot data environment must match the live data environment

    Piloting on cleaned or curated data subsets produces results that do not transfer to production. The pilot should be run on the actual data the production deployment will consume, including its inconsistencies, its update lag, and its governance gaps. If that data is not pilot-ready, the production environment is not ready either.

  • Four months of data readiness work before a pilot is not delay, it is sequencing

    Organizations that invest in data readiness before selection run faster, more defensible deployments. The pilot budget is spent once. The readiness work determines whether that spend produces a foundation or a postmortem.

Private Consultation

Bring these ideas into the room.

If this essay sounds like the conversation you're sitting with, Jessica responds personally to every inquiry.