Setting a Code Environment
Applies to DSS 4.1 and above | December 06, 2017
This How-To describes how to set Python or R code environments for plugins, projects and recipes (or other objects within the project).
Dataiku DSS code environments address the problem of managing dependencies and versions when writing code in R and Python. They are similar to the Python virtualenv. Code environments provide a number of benefits, including:
- Isolation. Two teams can work independently on different projects using different versions of Python (or R) and a set of libraries whose versions differ.
- Reproducibility of results. When you create a project bundle or API service package and push it to production, Dataiku DSS includes the specification for the project’s code environment, and then rebuilds the code environment according to that specification when you import the bundle into the Dataiku Automation node or the package into the Dataiku API node. In this way, environments are versioned on your production server and you can rollback your code to a previous version together with its code environment.
Creating New Code Environments
Your Dataiku DSS administrator, or any user in a group with the proper permissions, can create different code environments, which are then available for you to use.
Using Code Environments
A separate code environment can be specified for many Dataiku objects, including:
Setting a Project-Level Code Environment
By default, projects use the Dataiku DSS environment when processing Python or R code. In an individual project, you can set a different code environment to be used when processing code within that project.
- From the main project page, go to Settings, then the Settings tab, then the Code recipes panel.
- Under Environment for Python Recipes, deselect Use DSS builtin Python env. A dropdown appears that allows you to select a different environment.
Similarly, you can select an R environment using the controls under Environment for R Recipes
If there is an environment you expect to see that is missing, contact your administrator. They may need to create a new code environment or give you permission to use an existing one.
Note: If you plan to use Visual Machine Learning, the project-level code environment must include the scipy, scikit-learn, jinja2 and xgboost packages.
Setting a Code Environment in a Recipe
By default, Python and R recipes use the project’s code environment. For each recipe, you can set a different code environment to be used when processing code within that recipe.
On the Advanced tab of a recipe, you can set the code environment using the Selection behavior dropdown.
Setting a Code Environment in a Notebook
By default, Python and R notebooks use the project’s code environment. For each notebook, you can set a different code environment to be used when processing code within that notebook.
You can set the code environment at notebook creation time, or by changing the kernel (Kernel > Change kernel from the notebook menus).
Setting a Code Environment in a Web App
By default, Python Bokeh and R Shiny web apps use the project’s code environment. For each web app, you can set a different code environment to be used when processing code within that web app.
While in Edit mode, on the Settings tab of a web app you can set the code environment using the Code env dropdown.
Setting a Code Environment in a R Markdown Document
By default, R markdown documents use the project’s code environment. For each document, you can set a different code environment to be used when processing code within that document.
While in Edit mode, on the Settings tab of a document you can set the code environment using the Code env dropdown.
Setting a Code Environment in a Plugin
The plugin developer defines the code environment specification as part of the plugin. After installing a plugin that contains a code environment definition, you are prompted to create a code environment for the plugin.
Setting a Code Environment in Visual Machine Learning
Dataiku Visual Machine Learning allows you to create custom models using Python, in addition to the built-in models. The Python code environment to be used for training those custom models can be set in Design mode on the Python environment panel
Setting a Code Environment in Other Dataiku Objects
You can insert Python code in custom scenario steps, custom scenario triggers, custom models, custom dataset probes and checks… there are a lot of places you can use custom code in Dataiku DSS!
For objects that are not focused on code, but accept custom code, the dropdown list is typically placed near the custom code.
