AI governance in practice what the gap between policy and deployment is actually costing you

Sixty-five percent of organizations are now regularly using generative AI, nearly double the adoption rate from the prior year. Yet only 21 percent report having enterprise-wide policies governing its use (McKinsey, 2024). That is not a future problem approaching on the horizon. It is the condition most organizations are operating inside right now, and the distance between those two numbers is where risk lives. Earlier this year, I convened a J. Caresse and Company executive roundtable on AI governance in practice. The room included leaders from cybersecurity, HR, solution architecture, data management, and consulting. What I heard was not theoretical. It was operational: governance frameworks being stood up after the fact, ownership fragmented across functions that do not fully communicate with one another, employees moving faster than policy, and agentic AI arriving before anyone has finished building the structures required to contain the tools that came before it. This post synthesizes what those leaders shared, paired with the external research that corroborates what they described firsthand.

Governance without a home

When I asked roundtable participants what AI governance means to them, the answers were strikingly different depending on where they sit. A transformation lead described it as a training and change management problem. A cybersecurity executive framed it as a tooling and access control challenge. An HR leader said it had landed, by default, in her function's lap. None of them were wrong. That is precisely the problem. Gartner found that through 2026, more than 80 percent of enterprises will have attempted to deploy AI without a defined governance framework, and that ownership conflicts between IT, legal, HR, and business functions represent the primary obstacle to operationalizing policy (Gartner, 2024). What roundtable participants described maps directly onto that finding. The structural pattern is consistent: a small AI council, often sitting under the CISO, charged with producing policy but without the cross-functional relationships required to enforce it. Deloitte's 2024 State of Generative AI in the Enterprise found that fewer than one in three organizations has a dedicated AI governance function with clear executive sponsorship, and that most governance activity remains concentrated in IT and legal, with limited participation from HR, finance, and line-of-business leaders (Deloitte, 2024). One COO at the roundtable put it precisely: HR, IT, legal, and business leads are all being asked to coordinate on a problem none of them fully control, and none of them have the full information set required to make good decisions alone. PwC's 2024 AI Jobs Barometer reinforces this: companies with effective AI governance are significantly more likely to have appointed cross-functional AI steering committees with representation across legal, HR, technology, and business operations, rather than relying on a single function to own policy (PwC, 2024). Building that cross-functional muscle is not a natural organizational motion. It requires intentional design, executive sponsorship, and explicit accountability structures that most organizations have not yet put in place.

Shadow AI is not a warning sign. It is already here.

Several participants described shadow AI as an existing operational reality, not a forward-looking risk. Employees across functions are using unapproved tools, including browser-based large language models, local models running on personal devices, and consumer-grade productivity applications, to perform work that involves sensitive company data. One cybersecurity executive resolved this by blocking all unapproved AI tools at the network layer, using monitoring platforms combined with internal tooling. Unauthorized usage dropped sharply. But other participants pushed back on that approach immediately. A solution architect described what happens when lockdown is imposed without cultural or organizational support: workers move sensitive client information to personal machines and run local models to avoid detection. The resulting data governance exposure is more severe than the one the lockdown was designed to prevent. The underlying driver, participants agreed, is not defiance. It is pressure. Employees are receiving explicit and implicit signals that they are expected to demonstrate AI-driven productivity gains, and when approved tools are unavailable or inadequate, they find alternatives. BCG's 2024 AI at Work report found that 78 percent of employees using AI at work are doing so with at least some tools not approved by their employer, and that restrictive policies without corresponding enablement programs correlate with higher rates of unsanctioned tool use, not lower ones (BCG, 2024). KPMG's 2024 Trusted AI survey found that 61 percent of employees believe their organizations have not given them adequate guidance on what AI use is and is not acceptable, and that ambiguity is a stronger predictor of risky behavior than intent (KPMG, 2024). This is the finding I keep returning to. The control layer matters, but it cannot substitute for the education and enablement layer. Organizations investing only in restriction are trading one category of risk for another, often a worse one.

The identity problem that agentic AI is about to make much larger

The conversation shifted meaningfully when participants moved from generative AI tools to employee-built agents and MCP connectors. A cybersecurity executive called this the frontier where governance frameworks are most underdeveloped and where risk is accumulating fastest. Employees at multiple organizations represented in the roundtable are now using AI-assisted development environments to build applications, automate workflows, and create agents that interact with internal systems, frequently without formal IT review. The governance question is no longer just what data employees are sending to external models. It is what autonomous processes employees are deploying inside the organization, under what identities, with what access permissions, and with what oversight. Participants identified identity-based access control as the correct architectural principle: an agent should operate within the access permissions of the user who created it, so that whatever data the user cannot see, the agent cannot see. A solution architect described the obstacle precisely. Standard MCP connectors do not enforce this requirement, and patchwork orchestration across multiple tools cannot reliably replicate user-level access controls. AI gateways, agent registries, and identity management frameworks are emerging as the primary mechanisms, but the tooling and the policies are still being defined in real time. Gartner projects that by 2028, agentic AI will autonomously make at least 15 percent of day-to-day work decisions, and that organizations without formal agent governance frameworks will face material operational and compliance exposure (Gartner, 2025). An HR leader in the roundtable framed this as an unresolved people question as much as a technical one: when an employee-built agent makes a consequential error or accesses data it should not have accessed, who is responsible? MIT Sloan's 2024 research on AI accountability found that organizations struggle most with assigning responsibility when autonomous systems produce adverse outcomes, and that the absence of formal agent registries is a primary contributor to accountability gaps (MIT Sloan, 2024). EY's 2024 Reimagining the Workplace report adds a clarifying data point: fewer than 15 percent of organizations have defined accountability structures for AI-generated decisions or autonomous agent actions (EY, 2024). The answer requires both technical controls and explicit organizational policy. Most organizations currently have neither.

IT versus the business, and why that conflict is accelerating

A structural tension between IT and business functions surfaced repeatedly across the roundtable. For decades, IT carried the governance and compliance obligations associated with enterprise technology. Business functions operated within delivery cycles and change management processes that IT designed and enforced. AI-assisted development tools have disrupted that model. Business units now have direct access to development capability, and they are exercising it. A data management lead described the business perspective clearly: teams that once waited weeks or months for IT to deliver a solution can now build one themselves in days. From the business side, that feels like liberation. From the IT side, it looks like an expanding field of uncontrolled black boxes carrying compliance exposure that IT will ultimately be asked to answer for. BCG's 2024 AI and the Future of Business Functions report found that business-led AI development is accelerating across finance, marketing, operations, and supply chain, and that IT functions in most organizations have not updated their governance models to account for this shift (BCG, 2024). A cybersecurity executive described the organizational response emerging across the market: large organizations are hiring AI leads directly inside business functions, creating finance heads of AI and operations heads of AI who sit outside IT but carry development accountability. HBR's 2024 analysis of AI organizational design found that embedding technical AI capability inside business functions improves deployment speed and relevance but creates new accountability gaps when those embedded roles lack formal connections to enterprise risk and compliance processes (HBR, 2024). The accountability question participants raised was direct: if a business-embedded AI lead builds something that causes a compliance failure, a data breach, or a reputational incident, who owns the consequence? IT will bear operational scrutiny. Legal will face regulatory exposure. HR will manage the personnel outcome. But the organizational lines of responsibility are not clear, and in most companies, they have not been deliberately designed. McKinsey's 2024 research on AI governance maturity found that organizations with the clearest accountability structures are those that have established formal AI review processes spanning IT, legal, business, and risk functions, rather than assigning ownership to any single team (McKinsey, 2024).

The training gap that most organizations are not closing fast enough

Roundtable participants reached consensus on one point without significant debate: existing change management frameworks, security training programs, and organizational education models are too slow and too narrow to address AI risk at the current pace of adoption. A data management leader offered a precise and instructive example. Most organizations have security awareness training that explicitly prohibits employees from uploading sensitive customer data to external web tools. That prohibition has existed for years. Yet those same employees are pasting sensitive data into cloud-based AI prompts without recognizing that the behavior violates the same principle. The training was never updated to address AI-specific behaviors, and in many organizations, it still has not been. Deloitte's 2024 Human Capital Trends report found that 67 percent of employees say they have received no formal training on responsible AI use, and that the majority of organizations are relying on general digital literacy programs rather than AI-specific behavioral guidance to fill the gap (Deloitte, 2024). The roundtable surfaced a second structural obstacle that I found particularly striking: workforce reductions. Multiple participants noted that the teams responsible for designing and delivering updated training at scale have themselves been reduced, creating a situation where the organization's capacity for change management is shrinking at the exact moment the demand for it is expanding. PwC's 2024 Workforce Hopes and Fears survey found that 40 percent of employees say AI-related job restructuring has reduced the capacity of remaining staff to absorb new processes and training (PwC, 2024). One coaching and consulting executive at the roundtable advocated for teaching AI as a language skill rather than a text skill, so that employees who are not technologists can build genuine capability rather than performing surface-level compliance with training requirements. This framing maps to HBR's 2024 research on AI change management, which found that organizations achieving the highest AI adoption quality are those treating AI literacy as a leadership competency, not a technical skill, and that framing AI as a judgment and communication capability significantly increases adoption across non-technical populations (HBR, 2024). KPMG's research supports the same conclusion: AI training programs tied to specific role-based use cases produce measurably higher behavioral change than general awareness campaigns (KPMG, 2024).

ROI, architecture, and the hype reality gap

Several participants challenged the premise that AI investment automatically produces measurable returns on a short timeline, and the conversation that followed was among the most substantive of the roundtable. A solution architect offered a framework that resonated across the group: the durable question is not efficiency but effectiveness. The goal is not to find a process and replace it with AI. The goal is to identify where friction actually exists in a process, trace its origins, and determine whether AI is genuinely the best instrument to remove it. Many organizations are doing the opposite: selecting AI as the solution before diagnosing the problem, which produces cost overruns, underwhelming results, and organizational fatigue. Gartner's 2024 Hype Cycle for Artificial Intelligence cautions that organizations in the peak of inflated expectations phase consistently underestimate the implementation costs associated with data architecture, governance infrastructure, integration, and change management, and that total cost of ownership for enterprise AI is typically two to three times the initial technology investment (Gartner, 2024). CFOs are pressing for ROI on AI investments, and participants described the difficulty of providing it. When foundational data architecture costs, governance infrastructure, change management, training, and ongoing maintenance are factored in, the return timeline extends well beyond what technology vendors and internal champions typically project. A data management consultant described clients beginning to realize that token costs alone are higher than anticipated, and that the workforce reductions made in anticipation of AI-driven efficiency gains have removed the institutional knowledge required to maintain and improve the systems those gains depend on. BCG's 2024 AI Value Creation report found that only 26 percent of organizations report capturing the full business value of their AI investments, with data readiness, integration complexity, and change management costs cited as the primary barriers (BCG, 2024). MIT Sloan's 2024 research on AI investment outcomes found that the organizations producing the highest returns from AI share a common practice: they begin with a rigorous diagnostic of friction points across their processes before selecting technology, rather than beginning with technology selection and working backward (MIT Sloan, 2024). The solution architect at the roundtable described this directly, noting that the most durable value in many enterprise processes lies not in automating a task but in removing the copy-paste handoffs and manual data transfers that occur between systems sold as integrated but never fully were. EY's 2024 AI Pulse survey found that 55 percent of AI projects that fail to produce expected returns are attributable to insufficient process analysis prior to implementation, not to technology failure (EY, 2024). The implication is direct: the diagnostic phase is not a delay. It is the investment that protects all subsequent investments.

What comes next

The conditions roundtable participants described are not transitional. They are structural, and they will intensify before they stabilize. Agentic AI will continue to proliferate inside organizations regardless of whether governance frameworks are ready to contain it. Business functions will continue to build and deploy AI capabilities outside traditional IT delivery cycles. Employees will continue to use whatever tools help them meet productivity expectations, approved or not. The organizations that navigate this period most effectively will be those that treat governance not as a compliance exercise but as an operational capability, one that requires cross-functional ownership, technical depth, behavioral training, architectural discipline, and honest assessment of where AI genuinely creates value versus where it creates the appearance of progress. For executives, the priority is not to achieve perfect governance before moving forward. That standard is not achievable at the current pace of AI development. The priority is to close the most consequential gaps: establishing clear cross-functional accountability, building identity-aware agent governance before agentic AI scales further, updating security and behavioral training to address AI-specific risks, and requiring rigorous process diagnostics before approving AI investments. J. Caresse and Company will continue convening executive leaders across functions to surface what is working, what is not, and what the field is learning in real time. The distance between policy and practice is closeable. Closing it requires the kind of candid, cross-functional conversation this roundtable demonstrated is both possible and necessary.