AI Art

Plugin information

With this plugin, you will be able to:

• Generate images from a prompt using Stable Diffusion
• Generate images from another image and a prompt using Stable Diffusion

How to set up

CUDA

A CUDA-capable GPU with at least 6 GB of VRAM is recommended. You can run the plugin without a GPU, but it will be significantly slower.

If you’re using a CUDA GPU, your GPU must support CUDA 11.6, and the NVIDIA drivers must be installed on the server. For installation instructions, see NVIDIA Driver Installation Quickstart Guide.

Download weights

Before you can use the plugin, you need to download pre-trained weights to a local managed folder. The plugin is designed to work with version 2 of the Stable Diffusion weights, although it may work with other versions and derivatives as well.

For example, if you run the below code in a notebook, it will download the Stability AI version 2.1 weights, and upload them to a new managed folder.

The code requires a code environment with the huggingface-hub package installed.

import tempfile

import dataiku
import huggingface_hub

# Name of the local folder that will be created
FOLDER_NAME = "my_weights"
# Hugging Face repository
REPO = "stabilityai/stable-diffusion-2-1"
# Revision of the repository
#
# If using the Stability AI weights, the "fp16" revision is recommended
# if using CUDA. Otherwise, use the "main" revision.
REVISION = "fp16"

client = dataiku.api_client()
project = client.get_default_project()

# Create the managed folder
folder = project.create_managed_folder(FOLDER_NAME)

with tempfile.TemporaryDirectory(dir=".") as temp_dir:
    # Download the weights to a temp local dir
    huggingface_hub.snapshot_download(
       repo_id=REPO,
       revision=REVISION,
       local_dir=temp_dir,
       local_dir_use_symlinks=False,
       # Skip large files that the plugin doesn't need
       ignore_patterns=["*.ckpt", "*.safetensors"],
    )

    # Upload the weights to the managed folder
    folder.upload_folder("/", temp_dir)

Using a folder that’s stored on the local filesystem is recommended. If the folder is stored on a remote connection (Amazon S3, Google Cloud Storage, etc), the weights will be downloaded to a temporary directory every time the recipe is run.

How to use

AI Art contains two methods for generating images: Text-to-Image Generation, and Text-Guided Image-to-Image Generation

Text-to-Image Generation

Text-to-Image Generation is used to generate images from a text prompt.

1. Create a Text-to-Image Generation recipe with your weights folder as the input.
2. Enter a text prompt in the Prompt field.
3. If you’re using the fp16 revision of the weights, be sure to check the Half precision field.

Text-Guided Image-to-Image Generation

Text-Guided Image-to-Image Generation is used to modify an existing reference image based on a text prompt.

1. Obtain a reference image that you want to use as a base, and upload it to a managed folder.
2. Create a Text-Guided Image-to-Image Generation recipe with your weights folder and your base-image folder as the inputs.
3. Enter a text prompt in the Prompt field.
4. Enter the path to your base image in the Base image field.
5. If you’re using the fp16 revision of the weights, be sure to check the Half precision field.