The Dataiku LLM Mesh

Enable choice among data and AI infrastructure providers, third-party agents, and tools while also addressing cost management, compliance, and security concerns with the Dataiku LLM Mesh.

Build Agentic Apps With an LLM Mesh

In this technical guide from O’Reilly and Dataiku, discover how the LLM Mesh addresses key challenges around adopting LLMs at scale.

Enforce a Secure Gateway

The LLM Mesh acts as a secure API gateway to break down hard-coded dependencies, managing and routing requests between applications and underlying services.

Dataiku partners with leading providers of large language models (LLMs), vector databases, and accelerated and containerized compute capabilities. This includes Hugging Face for serving production-grade local LLMs easily and efficiently.

SEE AVAILABLE LLM CONNECTIONS

boxes showing all the AI services with which Dataiku LLM Mesh connects

diagram showing multi-LLM strategy with the Dataiku LLM mesh

Swap Underlying LLMs Without Rearchitecting

Should you build your GenAI applications or AI agents on top of today’s best model? Or should you hold out just a little longer until a new one emerges that’s more accurate, powerful, or better suited for your use case?

With Dataiku, you don’t have to choose. Balance trade offs between cost, performance, privacy, security, regulatory requirements, and more with a multi-LLM strategy. The LLM Mesh is the most comprehensive and agnostic LLM gateway offering on the market, partnering with the top Generative AI players for secure access to thousands of LLMs (including third-party or self-hosted).

We compared five different LLM models, something we couldn’t do without Dataiku, and found that one model continued to outperform the rest by a healthy margin for our specific use case.

We realize that no single LLM is good for every use case.

The Dataiku LLM Mesh offers substantial strategic benefits, notably accelerating time to market for GenAI chatbot development.

Deepthi Sanam

Group Lead of Data Engineering at Novartis

We were able to identify, kill off, or graduate good or bad ideas at a much more rapid pace than we were previously because of the very excellent abstractions that Dataiku provides through the visual recipes and through the LLM Mesh.

Kenan Yates

Group Product Manager at System1

Avoid Runaway LLM Usage Costs

Analyze up-to-the-minute data on LLM usage via Dataiku Cost Guard to ensure GenAI and AI agent applications aren’t breaking the bank.

A fully auditable log of who is using which model or service and for what purpose allows for cost tracking and internal re-billing.

DIG DEEPER ON LLM GUARD SERVICES

man sitting at desk with computer screen opening showing Dataiku Cost Guard with LLM Guard Services cost monitoring dashboards

screenshot showing Dataiku Safe Guard as part of the Dataiku LLM Mesh

Ensure Sensitive Data Stays in Your Control

When it comes to screening GenAI or agentic applications for private data or inappropriate content, the LLM Mesh — via Dataiku Safe Guard — can evaluate requests and responses for sensitive information. This includes detecting personally identifiable information (PII), toxic content, or forbidden terms.

The LLM Mesh then takes appropriate action: redact this sensitive information before sending the request to the LLM API, block the request entirely, and/or alert an admin.

LEARN MORE ABOUT LLM GUARD SERVICES

Standardize Quality Control

Dataiku Quality Guard provides standardized, enterprise-grade quality assurance for LLMs and Generative AI applications.

Automatically compute standard evaluation metrics like LLM-as-a-judge techniques (answer relevancy, answer correctness, context precision, etc.) as well as statistical techniques (BERT, Rouge and Bleu, etc.).

DISCOVER LLM GUARD SERVICES

screenshots of Dataiku product showing LLM Evaluation recipe as part of LLM Guard Services and Quality Guard

The Dataiku LLM Mesh provides a way to utilize LLMs in a controlled and auditable manner, which allows us to ensure that the models used by the data scientists are validated from a legal perspective.

Data Science Lead at an Energy Company

Forrester: The Total Economic Impact™ Of Dataiku

"Evaluating different prompts and estimating token costs using/comparing different LLMs was easy on the Dataiku platform."

Andres Buser

Chapter Lead New Modality IP at Roche

screenshot showing how organizations can keep model documentation up to date with Dataiku LLM Registry

Keep Pace With Regulatory Requirements

Qualifying, documenting, and framing your organization’s use of LLMs is a critical step toward regulatory readiness.

Keep model documentation up to date as well as rationalize which models should (or should not) be used for what use cases with the Dataiku LLM Registry.

MORE ABOUT AI GOVERNANCE WITH DATAIKU

Monitor Performance at Scale

The LLM Mesh monitors the round-trip performance for LLM services and providers so teams can diagnose issues and select the optimal service based on application needs and SLAs.

Additionally, caching of responses to common queries avoids the need to regenerate responses, offering both cost savings and a performance boost.

screenshot showing The LLM Mesh monitors the round-trip performance for LLM services and providers

screenshots showing Dataiku's GenAI builder components including LLM powered text recipes, code-free RAG

Build GenAI & Agents Quickly (& Safely)

Dataiku brings not only the LLM Mesh, but all the tools required for builders across the business to bring GenAI and AI agents to life — fast. .

The Universal AI Platform™ provides your stakeholders with everything they need to create GenAI applications and AI agents at scale.

DISCOVER GENAI & AGENT BUILDER TOOLS