howto

Setting a Code Environment

Applies to DSS 4.1 and above | December 06, 2017

This How-To describes how to set Python or R code environments for plugins, projects and recipes (or other objects within the project).

Dataiku DSS code environments address the problem of managing dependencies and versions when writing code in R and Python. They are similar to the Python virtualenv. Code environments provide a number of benefits, including:

  • Isolation. Two teams can work independently on different projects using different versions of Python (or R) and a set of libraries whose versions differ.
  • Reproducibility of results. When you create a project bundle or API service package and push it to production, Dataiku DSS includes the specification for the project’s code environment, and then rebuilds the code environment according to that specification when you import the bundle into the Dataiku Automation node or the package into the Dataiku API node. In this way, environments are versioned on your production server and you can rollback your code to a previous version together with its code environment.

Creating New Code Environments

Your Dataiku DSS administrator, or any user in a group with the proper permissions, can create different code environments, which are then available for you to use.

Using Code Environments

A separate code environment can be specified for many Dataiku objects, including:

Setting a Project-Level Code Environment

By default, projects use the Dataiku DSS environment when processing Python or R code. In an individual project, you can set a different code environment to be used when processing code within that project.

  1. From the top navigation bar, go to … > Settings, then the Code env selection panel.
  2. Under Environment for Python Recipes, deselect Use DSS builtin Python env. A dropdown appears that allows you to select a different environment.

Similarly, you can select an R environment using the controls under Environment for R Recipes

If there is an environment you expect to see that is missing, contact your administrator. They may need to create a new code environment or give you permission to use an existing one.

Note: If you plan to use Visual Machine Learning, the project-level code environment must include the scipy, scikit-learn, jinja2 and xgboost packages.

Setting a Code Environment in a Recipe

By default, Python and R recipes use the project’s code environment. For each recipe, you can set a different code environment to be used when processing code within that recipe.

On the Advanced tab of a recipe, you can set the code environment using the Selection behavior dropdown.

Setting a Code Environment in a Notebook

By default, Python and R notebooks use the project’s code environment. For each notebook, you can set a different code environment to be used when processing code within that notebook.

You can set the code environment at notebook creation time, or by changing the kernel (Kernel > Change kernel from the notebook menus).

Setting a Code Environment in a Web App

By default, Python Bokeh and R Shiny web apps use the project’s code environment. For each web app, you can set a different code environment to be used when processing code within that web app.

While in Edit mode, on the Settings tab of a web app you can set the code environment using the Code env dropdown.

Setting a Code Environment in a R Markdown Document

By default, R markdown documents use the project’s code environment. For each document, you can set a different code environment to be used when processing code within that document.

While in Edit mode, on the Settings tab of a document you can set the code environment using the Code env dropdown.

Setting a Code Environment in a Plugin

The plugin developer defines the code environment specification as part of the plugin. After installing a plugin that contains a code environment definition, you are prompted to create a code environment for the plugin.

Setting a Code Environment in Visual Machine Learning

Dataiku Visual Machine Learning allows you to create custom models using Python, in addition to the built-in models. The Python code environment to be used for training those custom models can be set in Design mode on the Python environment panel

Setting a Code Environment in Other Dataiku Objects

You can insert Python code in custom scenario steps, custom scenario triggers, custom models, custom dataset probes and checks… there are a lot of places you can use custom code in Dataiku DSS!

For objects that are not focused on code, but accept custom code, the dropdown list is typically placed near the custom code.