Skip to main content
Running Datalab’s containers requires a Google Cloud service account key to pull the container image. If the terms of your agreement require a license, we’ll also provide a license key.

​
License-enabled containers

Copy your license key, download the service account key, and run the script in this Github repository to get up-and-running:

​
Kubernetes deployment (Helm)

A Helm chart is available for deploying the container on Kubernetes clusters. Contact support@datalab.to to receive the chart and values reference for your deployment.

​
Fully-airgapped containers

A license key is not required to run a fully-airgapped container. If the terms of your agreement require a fully-airgapped container, we will provide:
  • Access to private registries that contain those images.
  • A Google Cloud service account key to pull images.
  • Directions for how to run the container.

​
www.datalab.to must be reachable

Our on-prem license requires that https://www.datalab.to is reachable in order to:
  1. Activate and register your license with our servers.
  2. Send usage metrics.

​
Usage data sent to Datalab

License activation and usage heartbeats do not send private data to Datalab. Our intent is to ensure compliance with our license and to easily support customers when they run into problems. The container sends the following to our servers:
  1. On container startup we activate your license. In that request, we send information about your hardware and OS available in /proc and /sys (in the container, not on your host).
  2. At regular intervals we send usage heartbeats that contain:
    1. The # of successful/failed inference requests completed since the last heartbeat
    2. The # of inference requests submitted to the container over a recent time window

​
I need a fully-airgapped deployment

We also support fully-airgapped deployments that do not require a license. Get started by filling out this form. Please reach out to us at support@datalab.to if you have questions.

​
Hardware Requirements

​
Throughput and Timeout Settings

For Chandra and Chandra-US containers that bundle vLLM, set these environment variables on the container before starting it: For chandra-us, GPU_MEMORY_UTILIZATION overrides the fraction of GPU memory reserved by vLLM. When unset, the GPU startup path chooses 0.8 on cards with up to 32 GB VRAM and 0.9 on larger cards. Pass variables through your deployment’s container environment configuration, then recreate or restart the container. Exporting them in your host shell only works if your launcher forwards them into the container. In Helm deployments with separate model services, configure the inference application and model server independently.
The page attempt budget is 1 + max(VLLM_ERROR_RETRIES, CHANDRA_REPETITION_MAX_RETRIES). Setting only VLLM_ERROR_RETRIES=0 still permits repetition retries. To limit a page to one inference attempt of about 30 seconds, set both retry values to 0 and VLLM_REQUEST_TIMEOUT_SECS=30. This is an inference-attempt bound, not an end-to-end request deadline; queueing and document preparation add time.

​
Health Check

Verify your container is running:
Expected response:

​
Next Steps

On-Prem API

API reference for the on-prem container image.

On-Prem Overview

Compare open-source and paid on-prem deployment options.

Cloud Quickstart

Try the cloud-hosted API for quick evaluation.

Error Codes

Understand HTTP error codes and troubleshooting steps.