Run it in Docker

The image curiosityai/spark is NVIDIA's CUDA runtime image, which carries cuBLAS, with the release archive in /app. It is built for linux/amd64 and linux/arm64.

docker run -d --name spark --gpus all -p 8080:8080 -v spark-data:/data \
  -e SPARK_ADMIN_PASSWORD='choose-a-password' curiosityai/spark:latest

The data, models and KV snapshots live in the volume mounted at /data. Keep it on a volume, or the library is gone when the container is re-created.

--gpus all needs the NVIDIA container toolkit on the host. Without it, or without a GPU, drop the flag and the models run on the CPU.

Docker Compose

services:
  spark:
    image: curiosityai/spark:latest
    restart: unless-stopped
    environment:
      SPARK_ADMIN_PASSWORD: ${SPARK_ADMIN_PASSWORD:?Set SPARK_ADMIN_PASSWORD}
    ports:
      - "8080:8080"
    volumes:
      - spark-data:/data
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

volumes:
  spark-data:

Remove the deploy section on a machine without an NVIDIA GPU.

Upgrade

docker compose pull && docker compose up -d

With docker run, pull the new image, remove the container and run it again with the same volume.

Use the models from other containers

Put Spark on the same Docker network as the container that calls it and use the container name as the host: http://spark:8080/v1. From a container to a Spark on the host, use the host's address and publish the port.