From shadow AI to operational infrastructure what small teams are building that enterprises have not yet figured out

Seventy-two percent of organizations reported using AI in at least one business function in 2024, up from 55 percent the year before. That number sounds like progress. It is not the whole story. The same McKinsey research found that fewer than 30 percent of those organizations had implemented meaningful risk mitigation practices for their most critical AI use cases (McKinsey, 2024). Adoption and readiness are not the same metric, and the gap between them is not a technology problem. It is a decision-making problem, and it is widening. Through a series of facilitated roundtable discussions I host at J. Caresse and Company, a counterintuitive pattern has emerged with enough consistency to name it: small, founder-led teams are building the operational AI infrastructure that enterprise organizations are still debating in steering committees. They are moving faster, spending less, and building more durable systems. The reason is not that they have better tools. It is that they are clearer about the problem they are solving before they reach for a tool at all. That clarity, what I think of as process thinking before AI, is the variable that separates organizations that benefit from AI from those that accumulate AI debt.

The real cost of moving too fast

Executive urgency is not the same as strategic momentum. One consultant in a recent roundtable described a client situation in which a CEO, prompted by a peer recommendation, issued a company-wide directive to identify and adopt AI tools within the quarter. No process audit had been conducted. No evaluation criteria existed. The result was fragmented adoption across departments, each team pursuing different platforms against different use cases, none of which were benchmarked against existing workflows. The technology was not the failure point. The decision architecture that preceded it was. This pattern is well documented at scale. McKinsey found that 40 percent of companies cite unclear business cases as a primary reason AI initiatives fail to scale beyond pilot programs (McKinsey, 2024). PwC's 2024 AI Business Survey found that organizations that rushed deployment without process alignment were three times more likely to report negative productivity impacts than those that invested in pre-implementation groundwork (PwC, 2024). Gartner projected that through 2025, more than 30 percent of AI projects would be abandoned after proof of concept, primarily due to poor data quality, unclear business value, and the absence of defined processes before deployment (Gartner, 2024). Deloitte's 2024 State of Generative AI in the Enterprise report captured the tension precisely: 79 percent of executives described urgency to scale AI, while only 22 percent felt their organizations were fully prepared to address the risks of doing so (Deloitte, 2024). The organizations closing this gap fastest are the ones that treat AI readiness as a process design question first, and a technology question second. The ones that do not are learning an expensive lesson about what happens when you automate ambiguity.

Accuracy risk and the missing baseline

One roundtable participant, a consultant working across marketing and operations functions, raised findings from a review conducted by a major international news organization. The review found that up to 40 percent of citations generated by leading large language models from verified news sources contained material inaccuracies: altered word choices, misattributed quotes, or subtle factual modifications that changed the original meaning. The individual errors were often minor in isolation. The risk is systemic and cumulative, particularly for any organization operating without a verified baseline of truth against which AI outputs can be checked. MIT Sloan Management Review has noted that hallucination rates in production AI systems remain a significant liability, particularly in regulated industries and in any workflow where outputs are consumed without human review (MIT Sloan, 2024). EY's 2024 AI Confidence Index found that 65 percent of business leaders expressed concern about AI accuracy and reliability, yet fewer than half had implemented any systematic validation process for AI-generated outputs (EY, 2024). KPMG research found that organizations lacking data governance frameworks were significantly more exposed to AI-generated errors propagating into customer-facing decisions (KPMG, 2024). The dimension that rarely surfaces in technical discussions of hallucination is this: the problem is significantly worse inside organizations that have not defined what accurate looks like in the first place. Deploying AI into a company with poorly documented processes does not surface existing errors. It accelerates and obscures them, pushing compounding inaccuracies downstream before anyone has the context to identify the original failure point. An AI readiness assessment is not a technology audit. It is a process audit. You cannot validate an output against a standard that does not exist.

Shadow AI and the knowledge bottleneck: two versions of the same problem

Two governance failures surfaced during a recent roundtable, and the group spent considerable time connecting them. The first was described by a product and operations leader who acknowledged that within his own business, he had become the primary bottleneck to every team function. He was building AI-driven workflows faster than he could document or transfer them. Operational knowledge lived in his head, not in shared systems, and his team was left with tools they could not fully use and processes they could not independently maintain. The second failure is its organizational equivalent: shadow AI, a condition in which team members use AI tools individually, with no governance, no shared context, and no visibility into how sensitive information is being handled. The term shadow AI echoes the earlier concept of shadow IT, and the analogy holds. Gartner estimated that shadow IT historically accounted for 30 to 40 percent of enterprise technology spending, much of it invisible to IT and security teams (Gartner, 2023). Shadow AI introduces the same opacity, with the added risk that employees may be passing sensitive business information into public large language model interfaces without organizational awareness. McKinsey noted that fewer than 35 percent of organizations had implemented policies governing employee use of external AI tools (McKinsey, 2024). That is not an AI problem. That is an AI behavioral change problem, and it requires the same change management infrastructure as any other shift in how people work. One participant described a company building a context-aware enterprise platform designed to unify AI tool usage across teams, ensuring that regardless of which model or interface an individual employee is using, the system maintains shared organizational context. The product is early, but the problem it targets is real and growing. HBR has argued that the next competitive differentiator in AI will not be which model a company uses, but how effectively the organization shares context across the people and systems using it (HBR, 2024). The knowledge-bottleneck problem and the shadow AI problem are structurally identical: both are symptoms of organizations that have not yet moved from task thinking to systems thinking.

What systems orchestration actually looks like in practice

The roundtable included a live demonstration that shifted the conversation from governance theory to operational architecture. A participant with deep product and go-to-market experience walked the group through a suite of autonomous agents he had configured and deployed across his business: lead sourcing and enrichment, campaign management, calendar preparation, LinkedIn content generation, inbox triage, and client briefing. Each agent operated from a shared cloud environment with its own compute resources and browser access, enabling persistent, session-independent operation across any device. The total infrastructure cost at the time of the demonstration was twenty dollars per month. This is not a novelty use case. It is a structural shift in what small teams can operationalize. BCG's 2024 research on AI agents found that autonomous agent deployments reduced task completion time by an average of 40 percent in knowledge-work environments, while freeing senior personnel for higher-order decision-making (BCG, 2024). Gartner predicted that by 2028, at least 15 percent of day-to-day work decisions would be made autonomously by AI agents, up from near zero in 2024 (Gartner, 2025). The participant connected this directly to the knowledge-bottleneck discussion: persistent cloud-based agents with their own compute represent a fundamentally different class of tool than single-session AI interfaces, because they can maintain context, execute tasks asynchronously, and operate while the human builder is engaged elsewhere. A second pattern emerged from separate participants who had independently built fully automated content production pipelines. Both described architectures that ingested raw video, applied editing and formatting rules, and distributed output across platforms with minimal human intervention. The convergence of two independent implementations using different model harnesses but identical structural logic is significant. It suggests that end-to-end automation is now accessible to individuals with moderate technical fluency, not just engineering teams with dedicated infrastructure budgets. McKinsey's 2024 analysis estimated that AI-assisted content production could reduce creation time by 50 to 60 percent while enabling a three-to-five times increase in content volume (McKinsey, 2024). The tools exist. The constraint is whether the organization has the process clarity to use them without creating new categories of risk.

Cross-model orchestration: the competency enterprises are still naming

The roundtable surfaced a practical approach to AI tool usage that contrasts sharply with the platform-loyalty narratives common in enterprise AI discussions. Participants described using multiple models in deliberate sequence, routing specific tasks to the model best suited for that step rather than committing all work to a single interface. One participant described drafting initial content structures in one model, then passing the output to a second model for slide deck construction, using each model's comparative strengths rather than forcing either to perform tasks outside its optimal range. The group demonstrated this live, building presentation materials and configuring agent workflows across different platforms within the same session. MIT Sloan has argued that the most effective enterprise AI implementations are model-agnostic by design, treating individual models as interchangeable components within a broader workflow architecture rather than as singular solutions (MIT Sloan, 2024). BCG's AI adoption research found that organizations using multi-model orchestration reported higher task quality scores and greater flexibility in adapting to new model releases than those locked into single-vendor deployments (BCG, 2025). KPMG's 2024 technology outlook identified orchestration capability as a core competency for AI-mature organizations, defining it as the ability to coordinate inputs, outputs, and context across multiple models, agents, and data sources in service of a unified business objective (KPMG, 2024). The roundtable demonstrated that this competency does not require enterprise infrastructure to practice. A small team with clear workflow logic, appropriate tool access, and a willingness to experiment across platforms can achieve orchestration outcomes that rival those of organizations with significantly larger AI budgets. BCG research found that companies with strong AI governance frameworks were 1.5 times more likely to report revenue gains from their AI investments than those without (BCG, 2024). The key variable is not spend. It is not even sophistication. It is intentionality: the organizational discipline to approach AI as a system to be designed, not a tool to be grabbed.

What comes next

The patterns documented across these roundtable conversations are early signals of a structural shift in how operational capability is distributed across organization sizes. Small, founder-led teams are reaching AI infrastructure maturity faster than many enterprises because they make decisions quickly, test without bureaucratic approval cycles, and build without legacy system constraints. The tools enabling this, persistent cloud-based agents, model context protocol connectors, cross-model orchestration frameworks, and automated content pipelines, are not experimental. They are in production use today, at costs measured in tens of dollars per month. The question for larger organizations is not whether to adopt these capabilities. It is how quickly they can build the governance, process clarity, and baseline accuracy frameworks that make adoption safe and scalable. That is the work of AI readiness assessment, of AI behavioral change programs, and of leadership teams willing to treat systems thinking as a prerequisite rather than an afterthought. The next phase of this work will require the market to mature in two parallel directions. Enterprise-grade context-awareness platforms will need to achieve product-market fit to address shadow AI and knowledge-bottleneck problems at scale. Simultaneously, every organization deploying agentic infrastructure today will need to develop verification disciplines that match the pace of their automation. The competitive advantage will belong to those who treat accuracy, governance, and orchestration not as constraints on AI adoption, but as the foundation that makes AI adoption durable. J. Caresse and Company will continue to facilitate these practitioner conversations and synthesize findings across the research landscape as both the tools and the organizational responses continue to evolve.