The Governance Blueprint CoEs Use to Scale Self-Service and AI Agents

At the intersection of innovation and responsibility lies a challenge every enterprise must navigate: how to scale AI and analytics without compromising on governance. In the fourth session of Dataiku's CoE webinar series, we explored how organizations can bring governance and self-service together through a harmonized strategy. The result? A smarter, safer, and faster route to enterprise AI.

Joining this session were two of Dataiku’s foremost experts: Jon Tudor, Director of Business Architecture, and Jacob Beswick, Senior Director of AI Governance Solutions. Together, they offered a deep dive into how CoEs are leveraging Dataiku, The Universal AI Platform™, to ensure governance is an enabler — not a blocker — to enterprise-wide self-service and the safe deployment of AI agents.

The Playground Analogy: Framing the Governance-Self-Service Dynamic

As Jon explained, governance is often perceived as a set of restrictions that slow progress, while self-service is viewed as the path of least resistance to outcomes. But these two are not inherently at odds. A CoE, much like a playground supervisor, exists to both enable freedom and ensure safety. Governance, then, is about making the easiest path the safest one through smart architectural design.

Architectural Governance: Where Process Meets Design

This concept of architectural governance is central to balancing scale and safety. Users will naturally take the fastest route to their goal. The CoE’s responsibility is to make sure that route is governed by design. In this session, we looked at how Dataiku helps organizations build governance directly into platform architecture, so users don’t even feel it — but benefit from it continuously.

Some examples of architectural governance in practice, powered by Dataiku, included:

• In-database or Kubernetes processing is enforced by default.
• Pre- and post-SQL statements automatically applied.
• Connection management to restrict or allow data access.
• Test-driven development using the Dataiku Project Assessment Tool (PAT).
• Support for user-installed libraries with guardrails (e.g., blacklisting risky or costly packages).
• Prompt routing and governance via the Dataiku LLM Mesh, which handles prompt safety, LLM selection, and cost control.

Each of these examples demonstrates how architectural design can deliver governance and safety without slowing down the user experience.

Getting Started: Governance by Design, Not by Exception

To implement architectural governance, Jon laid out a clear method:

1. Identify the external regulations and internal policies that apply (especially for evolving AI and agentic AI use cases).
2. Determine which user behaviors and data practices need oversight.
3. Brainstorm controls and identify which can be automated through architecture.
4. Score each control by effectiveness, ease of use, automation potential, and implementation effort.
5. Prioritize the right set of controls and begin implementation.

The goal is to bake governance into the platform, not apply it after the fact.

Operationalizing Data Products: Governance at the Speed of Innovation

The next section focused on release governance. Traditional governance workflows can increase cycle time and frustrate users, especially when launching new data products or AI agents. With Dataiku Govern and the Project Assessment Tool, teams can streamline these release processes without compromising compliance or oversight.

Key questions CoEs must address:

• Who can operationalize data products (and does this change with AI agents)?
• What technical standards must be met?
• When are reviews needed, and who conducts them?
• Can low-risk projects bypass reviews, and should high-risk ones have added scrutiny?
• What metadata and documentation are required?

Automation is critical. By integrating standards into test-driven checks, users can receive instant feedback and self-correct before pushing to production. This creates a seamless path from build to deployment, with governance built in.

Defining & Scaling AI Governance: Frameworks, Roles, & Readiness

Next, Jacob expanded on how organizations can formalize AI governance beyond just machine learning pipelines. AI governance is about the orchestration and enforcement of processes, rules, and requirements that align AI initiatives with business goals and risk frameworks. It covers both traditional and GenAI systems, including AI agents.

Jacob outlined five governance readiness foundations that organizations can prepare when planning to implement AI governance:

1. Framework Alignment (e.g., internal policy or EU AI Act): Whether a formal framework has been selected and understood across the organization.
2. Leadership Sponsorship: The presence of executive buy-in, strategy, and sponsorship for governance initiatives.
3. Clear Responsibilities & Ownership: A well-defined responsible, accountable, consulted, and informed (RACI) model where individuals know their roles (e.g., reviewer, signatory, process owner) and responsibilities.
4. Defined Policies & Processes: Governance policies that are consistently and coherently implemented.
5. Tooling for Operationalization: Systems like Dataiku Govern used to translate policies into scalable, repeatable governance workflows.

He emphasized that every AI governance framework must be operationalized. Having a policy is only the beginning; what matters is enabling those policies through tooling, repeatable workflows, and integrated responsibilities across teams. He also noted that immature AI governance often involves scattered team-level efforts, which can lead to misalignment and redundancy.

Ethical AI by Design: The Dataiku LLM Mesh & Agent Governance

Finally, the session wrapped with a look at the Dataiku LLM Mesh, a middleware layer that manages GenAI prompt flows and ensures enterprise-ready routing, compliance, and cost management. With the Dataiku LLM Mesh, organizations can:

• Route prompts to approve LLMs.
• Block or alert on cost overruns using the Dataiku Cost Guard.
• Monitor toxicity and ethical risks.
• Check for faithfulness and relevance via supplemental agent evaluation.

Jacob and Jon explained how this setup enables organizations to enforce regulatory requirements like GDPR, CCPA, HIPAA, and the EU AI Act — not only during development but also in ongoing production use. They cautioned against relying solely on human intervention, advocating instead for automation and observability to manage governance complexity at scale.

Final Takeaway: Make Governance Seamless & Scalable

Both Jon and Jacob underscored the same message: Governance should not be a blocker, and self-service should not be a risk. With the right architectural governance, automation, and tools from Dataiku — like Dataiku Govern, PAT, and the LLM Mesh — CoEs can make the safest path the easiest one.

To learn more, catch the replay of this webinar and explore how your team can architect scalable, responsible AI with Dataiku, The Universal AI Platform™.