muna deploy

The deploy command deploys a compiled model to your GPU fleets, whether on-prem or from third-party GPU platforms. Compiled models can yield up to 45x reductions in cold-start times, along with reduced latency and higher utilization. See why. Each deployment runs an OpenAI-compatible web server, which then forwards requests to the compiled model.

Usage

$ muna deploy [OPTIONS] TAG --provider PROVIDER

Argument	Description
`TAG`	Compiled model tag (e.g. `@google/gemma-4-26b-a4b-it`)

Options

Option	Description
`--provider`	Cloud to deploy to: `modal`, `baseten`, `spheron`, or `baremetal`. Required.
`--gpu`	GPU: `a100`, `h100`, `h200`, or `b200`.
`--gpu-count`	Number of GPUs to request.
`--cpu`	Number of vCPUs to request.
`--memory`	Memory to request, in MB.
`--name`	Deployment name.
`--concurrency`	Maximum concurrent requests sent to an instance.
`--min-replicas`	Minimum replicas for autoscaling.
`--max-replicas`	Maximum replicas for autoscaling.
`--scaledown-window`	Autoscaling scale down window, in seconds.
`--ssh-host`	SSH target for `baremetal` deployments, e.g. `'root@1.2.3.4 -p 22 -i ~/.ssh/key'`.
`--endpoint-url`	Public base URL where the deployed server is reachable. Required for `baremetal`.
`--wait`	Wait until the deployment is complete.

Deploy a compiled model to Modal:

# 🚀 Deploy a compiled LLM to Modal
$ muna deploy @google/gemma-4-26b-a4b-it --provider modal --gpu b200

This command will create a lightweight app on Modal that runs the web server.

This command requires the modal package to be installed. Run pip install modal.

Deploying to Baseten

Deploy a compiled model to Baseten:

# 🚀 Deploy a compiled LLM to Baseten
$ muna deploy @google/gemma-4-26b-a4b-it --provider baseten --gpu b200

This command will create and deploy a lightweight service on Baseten that runs the web server.

This command requires the truss package to be installed and logged in. Run pip install truss, then truss login.

Deploying to Spheron

Deploy a compiled model to Spheron:

# 🚀 Deploy a compiled LLM to Spheron
$ muna deploy @google/gemma-4-26b-a4b-it --provider spheron --gpu b200

This command requires a Spheron API key in the SPHERON_API_KEY environment variable, and a --gpu.

Deploying to Your Own Servers

Deploy a compiled model to any server you can SSH into:

# 🚀 Deploy a compiled LLM to your own GPU server
$ muna deploy @google/gemma-4-26b-a4b-it \
    --provider baremetal \
    --ssh-host "root@1.2.3.4 -p 22 -i ~/.ssh/key" \
    --endpoint-url https://gemma.example.com

The web server listens on port 8000. Use --endpoint-url to tell Muna the public URL that routes to it.

CLI Reference

Compilation

Inference

Deployment

Account

Managing Compiled Models

Usage

Options

Deploying to Baseten

Deploying to Spheron

Deploying to Your Own Servers

​Usage

​Options

​Deploying to Modal

​Deploying to Baseten

​Deploying to Spheron

​Deploying to Your Own Servers

Usage

Options

Deploying to Modal

Deploying to Baseten

Deploying to Spheron

Deploying to Your Own Servers