Compiling Models - Muna Docs

Muna compiles stateless Python functions into self-contained executable binaries.

Your Python code is lowered directly to native code, and does not rely on the Python runtime (or any other managed runtime).

Why Compile?

Deployment is not the first step in shipping a model—compilation is. Compiled models are smaller, faster, and can yield up to 45x reductions in cold-start times:

Cumulative time to first token, compiled vs. uncompiled.

This matters most in serverless and autoscaling deployments, where every scale-up event pays the cold-start tax. Faster cold starts mean fewer warm replicas—and a smaller GPU bill. A compiled model is also a portable artifact: the exact same binary can serve requests on our hosted endpoint, on your own GPUs, or on end-user devices—with hardware-optimized performance on each.

Defining a Function

Write your function, add a docstring, then wrap it with our @compile decorator:

greeting.py

from muna import compile

@compile()
def greeting(name: str) -> str:
    """
    Say a friendly greeting.
    """
    return f"Hey there {name}! We're glad you're using Muna and we hope you like it."

The docstring is used as a description, is required, and must be 100 characters or less.

The function must specify parameter and return type annotations. Learn more.

Compiling the Function

Use the Muna CLI to compile the function. First, make sure you are logged into the Muna CLI:

# Login to the CLI
$ muna auth login <access key>

Then run the following command:

# Compile the function
$ muna compile --overwrite greeting.py

Muna will upload your code, perform code generation on your function and its dependencies, then compile for our supported platforms:

Compiling a function with the Muna CLI. This is realtime--not sped up.

Using the Function

Depending on the complexity of your function, it can take anywhere from a few seconds to a few minutes for the function to be compiled for all platforms. Once the function is compiled, you can run it everywhere:

import { Muna } from "muna"

// 💥 Create your Muna client
const muna = new Muna({ accessKey: "..." });

// 🔥 Run inference
const prediction = await muna.predictions.create({
    tag: "@your-username/greeting",
    inputs: { name: "Lina" }
});

// 🚀 Print the result
console.log(prediction.results[0]);

from muna import Muna

# 💥 Create your Muna client
muna = Muna(access_key="...")

# 🔥 Run inference
prediction = muna.predictions.create(
    tag="@your-username/greeting",
    inputs={ "name": "Lina" }
)

# 🚀 Use the results
print(prediction.results[0])

import Muna

// 💥 Create a Muna client
let muna = Muna(accessKey: "...")

// 🔥 Run inference
let prediction = try await muna.predictions.create(
    tag: "@your-username/greeting",
    inputs: ["name": "Lina"]
)

// 🚀 Use the results
print(prediction.results![0])

import ai.muna.muna.Muna

// 💥 Create a Muna client
val muna = Muna("...")

// 🔥 Run inference
val prediction = muna.predictions.create(
    "@your-username/greeting",
    mapOf("name" to "Lina")
)

// 🚀 Use the results
println(prediction.results!![0])

using Muna;

// 💥 Create your Muna client
var muna = new Muna(accessKey: "...");

// 🔥 Run inference
var prediction = await muna.Predictions.Create(
    tag: "@your-username/greeting",
    inputs: new() { ["name"] = "Lina" }
);

// 🚀 Use the results
Debug.Log(prediction.results[0]);

# 🔥 Run inference
$ muna predict @your-username/greeting --name Lina

You can check the compilation status of the function at muna.ai/predictors.

​Why Compile?

​Defining a Function

​Compiling the Function

​Using the Function

Why Compile?

Defining a Function

Compiling the Function

Using the Function