Plugin information

NVIDIA NIM – Generative AI Plugin

With this plugin, you can leverage and deploy NVIDIA NIM LLMs.

Plugin Components

The plugin provides the following two components:

1. Dataiku LLM Mesh connection
A custom LLM Mesh connection that provides the following capabilities:

• to connect NVIDIA NIM Text/Multimodal Chat Completions and Embedding Models via Dataiku LLM Mesh
• supports streaming and tool calling for compatible NIM models

The plugin is agnostic regarding the deployment location for the NIM LLM; for example, LLMs can be hosted in NVIDIA Cloud, self-hosted using the deployment macro provided by this plugin, or hosted elsewhere.

2. NIM Deployment Macro
The NIM deployment macro provides the following capabilities:

• deploy, list and remove the NVIDIA GPU Operator and NIM Operator
• deploy, list and remove NVIDIA NIM Services

Note: it is not mandatory to use the macro to deploy the GPU and NIM Operators. In fact, in some instances it is preferable (or even necessary) to deploy the GPU and NIM Operators externally (for example, using the OpenShift OperatorHub) instead of using the provided deployment action.

Limitations and Prerequisites

Common pre-requisites:

• Dataiku >= v13.4.0
• An NVIDIA AI Enterprise AI corporate subscription, or an NVIDIA Developer Program membership

NIM Deployment pre-requisites:

• NIM Container Registry and NIM Model Repository credentials (which can both be an NVIDIA Authentication API Key if using NGC)
• An attached Kubernetes cluster with:
  • An auto-provisioning Storage Class that supports “ReadWriteMany” access mode (see NIM Operator docs).
  • (Optional) The Prometheus operator installed on the cluster, only required if leveraging horizontal pod autoscaling
  • (Optional) The Nginx ingress controller installer on the cluster; without this, the deployment macro defaults to exposing the NIM Services using a NodePort Kubernetes service.

Plugin Installation and Configuration

Install the Plugin using the installation guide. Once installed, configure the appropriate plugin presets.

NIM API Keys Preset

This preset stores per-user NIM API keys. Use this preset when the NIM LLM endpoints require an API key authentication, such as when using NIM hosted on NVIDIA Build.

• Go to Plugins → Installed → NVIDIA NIM Plugin → Settings → NIM API Keys → Add a Preset.
  
• Every user can set up the preset under their Dataiku profile → API Credentials → Credentials → Plugin credentials. Click the Edit icon and paste your NIM API key.
  

Self-hosted credentials Preset

This preset stores NIM Container Registry and NIM Model Repository credentials. Use this preset when self-hosting NIMs on an attached Kubernetes cluster using the NIM Deployment Macro.

• Go to Plugins → Installed → NVIDIA NIM Plugin → Settings → Self-hosted credentials → Add a Preset.
• For Docker container registry, enter host, username and API key
• For NIM model registry
• If using NGC, simply enter the NGC API key,
• If using an alternative model registry (such as JFrog), select the ‘override model registry’ checkbox, and enter model registry the host, protocol and API key.

NIM Environment Variables Preset

This preset provides a mechanism to override the values of NIM environment variables. It should only be used when self-hosting NIMs on an attached Kubernetes cluster using the NIM Deployment Macro.

• Go to Plugins → Installed → NVIDIA NIM Plugin → Settings → NIM Environment Variables → Add a Preset.
• Override the value of any NIM environment variable

NIM LLM Mesh Connection

• Go to Administration → Connections → New Connection → Custom LLM (LLM Mesh)

• Provide a connection name and select NVIDIA NIM Connector in the Plugin dropdown

• To add models – Click Add Model
  • Id: Unique name to identify model
  • Capability : Chat Completion / Text Embedding
  • Type: NVIDIA NIM LLM Connector
  • Keys Preset: Select the preset name defined in the plugin setting
  • Endpoint URL: Provide complete URL (examples below)
  • For chat completion – https://hostname/v1/chat/completions
  • For chat completion – https://hostname/v1/retrieval/nvidia/embeddings
  • Model Key: Provide the model key accepted by NIM containers. E.g. ‘google/gemma-2b’ or ‘NV-Embed-QA’
  • Input Type (This property only applied to Embedding models. By default, it is set to query): query or passage

Once the setup is complete, you can access NIM models both in LLM Powered Visual Recipes, Prompt Studios and using Python and REST LLM Mesh APIs.

NIM Service Deployment

The NIM deployment macro is located in the Adminsitration → Clusters → *Cluster name* → Actions tab of the Kubernetes cluster.

GPU and NIM Operators

The first three macro actions provide the option to list, deploy and remove the NVIDIA GPU and NIM Operators. If these Operators are not already available onto the cluster, they must be deployed prior to deploying your first NIM Serivce.

NIM Services

The next three macro actions provide the option to list, deploy and remove NVIDIA NIM Services. Under the hood, Dataiku leverages the NVIDIA NIM Operator, so all the options presented in the UI are simply those descripted in the NIM Operator documentation.

• To deploy a NIM Service, first use the “NIM Service: Add” macro:

• Then, use the “NIM Service: Inspect” macro to check the deployment status, and retrieve the model endpoint once the Service has been successfully rolled out. The model endpoint can then be used to create a NIM LLM Mesh connection.

Once the setup is complete, you can access models both in LLM Powered Visual Recipes, Prompt Studios and using Python and REST LLM Mesh APIs

Plugin Demo

Advancing healthcare with Dataiku and Nvidia

Curious how Dataiku and Nvidia partnership is advancing healthcare? Checkout the following blog post:

Advancing healthcare with Dataiku and Nvidia