Using the Muna Client

Muna’s native client, muna.predictions, runs any compiled model locally.

Creating Predictions

import { Muna } from "muna"

// 💥 Create your Muna client
const muna = new Muna({ accessKey: "..." });

// 🔥 Run inference locally
const prediction = await muna.predictions.create({
    tag: "@fxn/greeting",
    inputs: { name: "Yusuf" }
});

// 🚀 Print the result
console.log(prediction.results[0]);

from muna import Muna

# 💥 Create your Muna client
muna = Muna(access_key="...")

# 🔥 Run inference locally
prediction = muna.predictions.create(
    tag="@fxn/greeting",
    inputs={ "name": "Muna" }
)

# 🚀 Use the results
print(prediction.results[0])

import { Muna } from "@muna/expo"

// 💥 Create your Muna client
const muna = new Muna({ accessKey: "..." });

// 🔥 Run inference locally
const prediction = await muna.predictions.create({
    tag: "@fxn/greeting",
    inputs: { name: "Yusuf" }
});

// 🚀 Print the result
console.log(prediction.results[0]);

import Muna

// 💥 Create your Muna client
let muna = Muna(accessKey: "...")

// 🔥 Run inference locally
let prediction = try await muna.predictions.create(
    tag: "@fxn/greeting",
    inputs: [ "name": "Terri" ]
)

// 🚀 Use the results
print("\(prediction.results![0]!)")

import ai.muna.muna.Muna

// 💥 Create your Muna client
val muna = Muna("...")

// 🔥 Run inference locally
val prediction = muna.predictions.create(
    "@fxn/greeting",
    mapOf("name" to "Timi")
)

// 🚀 Use the results
println(prediction.results!![0])

using Muna;

// 💥 Create your Muna client
var muna = new Muna(accessKey: "...");

// 🔥 Run inference locally
var prediction = await muna.Predictions.Create(
    tag: "@fxn/greeting",
    inputs: new() { ["name"] = "Peter" }
);

// 🚀 Use the results
Debug.Log(prediction.results[0]);

use std::collections::HashMap;
use muna::{Muna, Value};

// 💥 Create your Muna client
let muna = Muna::new(Some("..."), None);

// 🔥 Run inference locally
let mut inputs = HashMap::new();
inputs.insert("name".to_string(), "Yusuf".into());
let prediction = muna.predictions.create(
    "@fxn/greeting",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();

// 🚀 Print the result
println!("{:?}", prediction.results.unwrap()[0]);

Streaming Predictions

Muna supports consuming the partial results of an inference request as they are made available by the compiled model:

// 🔥 Create an inference stream
const stream = await muna.predictions.stream({
  tag: "@text-co/split-sentence",
  inputs: { text: "Hello world" }
});

// 🚀 Consume the stream
for await (const prediction of stream)
  console.log(prediction.results[0]);

# 🔥 Create an inference stream
stream = muna.predictions.stream(
  tag="@text-co/split-sentence",
  inputs={ "text": "Hello world" }
)

# 🚀 Consume the stream
for prediction in stream:
  print(prediction.results[0])

// 🔥 Create an inference stream
const stream = await muna.predictions.stream({
  tag: "@text-co/split-sentence",
  inputs: { text: "Hello world" }
});

// 🚀 Consume the stream
for await (const prediction of stream)
  console.log(prediction.results[0]);

// 🔥 Create an inference stream
let stream = try await muna.predictions.stream(
  tag: "@text-co/split-sentence",
  inputs: ["text": "Hello world"],
)

// 🚀 Consume the stream
for try await prediction in stream {
  print("\(prediction.results?[0])")
}

// 🔥 Create an inference stream
val stream = muna.predictions.stream(
  "@text-co/split-sentence",
  mapOf("text" to "Hello world")
)

// 🚀 Consume the stream
stream.use {
  for (prediction in it.consume())
    println(prediction.results!![0])
}

// 🔥 Create an inference stream
var stream = await muna.Predictions.Stream(
  tag: "@text-co/split-sentence",
  inputs: new() { ["text"] = "Hello world" }
);

// 🚀 Consume the stream
await foreach (var prediction in stream)
  Debug.Log(prediction.results[0]);

use std::collections::HashMap;
use futures_util::StreamExt;

// 🔥 Create an inference stream
let mut inputs = HashMap::new();
inputs.insert("text".to_string(), "Hello world".into());
let mut stream = muna.predictions.stream(
  "@text-co/split-sentence",
  inputs,
  None,
).await.unwrap();

// 🚀 Consume the stream
while let Some(Ok(prediction)) = stream.next().await {
  println!("{:?}", prediction.results.unwrap()[0]);
}

Consuming Inference Streams

Streaming in Muna is designed to fully separate how a compiled function is implemented from how the function might be consumed. Consider these two functions:

def predict() -> str:
    return "hello from Muna"

def predict() -> Iterator[str]:
    yield "hello"
    yield "hello from"
    yield "hello from Muna"

Here are the results of creating vs. streaming each function at runtime:

Using predictions.create with eager.py

In this case, the single result is returned:

// Run inference with the eager function
const prediction = await muna.predictions.create({
  tag: "@muna/eager",
  inputs: { }
});

// Display the results
console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"

Using predictions.create with generator.py

In this case, the Muna client will consume all partial results yielded by the function then return the very last one:

// Run inference with the generator function
const prediction = await muna.predictions.create({
  tag: "@muna/generator",
  inputs: { }
});

// Display the results
console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"

Using predictions.stream with eager.py

In this case, the Muna client will return an inference stream with the single result returned by the function:

// Create an inference stream with the eager function
const stream = await muna.predictions.stream({
  tag: "@muna/eager",
  inputs: { }
});

// Display the results
for await (const prediction of stream)
  console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"

Using predictions.stream with generator.py

In this case, the Muna client will provide an inference stream containing all partial results yielded by the function:

// Create an inference stream with the generator function
const stream = await muna.predictions.stream({
  tag: "@muna/generator",
  inputs: { }
});

// Display the results
for await (const prediction of stream)
  console.log(prediction.results[0]);

// Outputs:
// "hello"
// "hello from"
// "hello from Muna"

You can choose how to consume a compiled function depending on what works best for your user experience. You don’t have to care about the underlying function!

Using Inference Values

Muna supports a fixed set of value types for inference input and output values:

Floating Point Values

Muna supports the following floating-point numbers:

Muna value type	C/C++ type	Description
`float16`	`float16_t`	IEEE 754 16-bit floating point number.
`float32`	`float`	IEEE 754 32-bit floating point number.
`float64`	`double`	IEEE 754 64-bit floating point number.

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        radius: 4.5
    }
});
const radius = prediction.results[0] as number;

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "radius": 4.5
    }
)
radius: float = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        radius: 4.5
    }
});
const radius = prediction.results[0] as number;

let prediction = try await muna.predictions.create(
    tag: "@fxn/identity",
    inputs: [
        "radius": 4.5
    ]
)
let radius = prediction.results![0] as! Float

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "radius" to 4.5f
    )
);
val radius = (Float)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new() {
        ["radius"] = 4.5f
    }
);
var radius = (float)prediction.results[0];

let mut inputs = HashMap::new();
inputs.insert("radius".to_string(), 4.5f32.into());
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Float(radius) = &prediction.results.unwrap()[0] else { unreachable!() };

In languages that don’t support fixed-size floating point scalars, the data type for floating point values defaults to float32. Use a tensor constructor to explicitly specify the data type.

Support for half-precision floating point scalars float16 is planned for the future depending on language support.

Muna supports floating point vectors (i.e. one-dimensional floating point tensors):

import type { Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new Float32Array([ 1.2, 2.2, 3.2, 4.5 ])
    }
});
const vector = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "vector": np.array([1.2, 2.2, 3.2, 4.5], dtype="float32")
    }
)
vector: np.ndarray = prediction.results[0]

import type { Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new Float32Array([ 1.2, 2.2, 3.2, 4.5 ])
    }
});
const vector = prediction.results[0] as Tensor;

import ai.muna.muna.types.Float32Tensor;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "vector" to floatArrayOf(1.2f, 2.2f, 3.2f, 4.5f)
    )
);
val vector = (Float32Tensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new() {
        ["vector"] = new [] { 1.2f, 2.2f, 3.2f, 4.5f }
    }
);
var vector = (Tensor<float>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("vector".to_string(), Value::Tensor(Tensor {
    data: TensorData::Float32(vec![1.2, 2.2, 3.2, 4.5]),
    shape: vec![4],
}));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(vector) = &prediction.results.unwrap()[0] else { unreachable!() };

Although Muna supports input vectors, compiled models will always output either scalars or Tensor instances—never plain vectors.

Muna supports floating point tensors:

import type { Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        matrix: {
            data: new Float64Array([ 1.2, 2.2, 3.2, 4.5 ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "matrix": np.array([ [1.2, 2.2], [3.2, 4.5] ], dtype="float64")
    }
)
matrix: np.ndarray = prediction.results[0]

import type { Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        matrix: {
            data: new Float64Array([ 1.2, 2.2, 3.2, 4.5 ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import ai.muna.muna.types.Float64Tensor;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "matrix" to Float64Tensor(
            doubleArrayOf(1.2, 2.2, 3.2, 4.5), // data
            intArrayOf(2, 2) // shape
        )
    )
);
val matrix = (Float64Tensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new () {
        ["matrix"] = new Tensor<double>(
            data: new [] { 1.2, 2.2, 3.2, 4.5 },
            shape: new [] { 2, 2 }
        )
    }
);
var radius = (Tensor<double>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("matrix".to_string(), Value::Tensor(Tensor {
    data: TensorData::Float64(vec![1.2, 2.2, 3.2, 4.5]),
    shape: vec![2, 2],
}));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(matrix) = &prediction.results.unwrap()[0] else { unreachable!() };

Integer Values

Muna supports several signed and unsigned integer scalars:

Muna value type	C/C++ type	Description
`int8`	`int8_t`	Signed 8-bit integer.
`int16`	`int16_t`	Signed 16-bit integer.
`int32`	`int32_t`	Signed 32-bit integer.
`int64`	`int64_t`	Signed 64-bit integer.
`uint8`	`uint8_t`	Unsigned 8-bit integer.
`uint16`	`uint16_t`	Unsigned 16-bit integer.
`uint32`	`uint32_t`	Unsigned 32-bit integer.
`uint64`	`uint64_t`	Unsigned 64-bit integer.

const prediction = await muna.predictions.create({
    tag: "@fxn/squeeze",
    inputs: {
        oranges: 12
    }
});
const cups = prediction.results[0] as number;

prediction = muna.predictions.create(
    tag="@fxn/squeeze",
    inputs={
        "oranges": 12
    }
)
cups: int = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/squeeze",
    inputs: {
        oranges: 12
    }
});
const cups = prediction.results[0] as number;

val prediction = muna.predictions.create(
    "@fxn/squeeze",
    mapOf(
        "oranges" to 12
    )
);
val cups = (Int)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/squeeze",
    inputs: new() {
        ["oranges"] = 12
    }
);
var cups = (int)prediction.results[0];

let mut inputs = HashMap::new();
inputs.insert("oranges".to_string(), 12i32.into());
let prediction = muna.predictions.create(
    "@fxn/squeeze",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Int(cups) = &prediction.results.unwrap()[0] else { unreachable!() };

When integer scalars are passed to compiled models, the data type defaults to int32. Use a tensor constructor to explicitly specify the data type.

Muna supports integer vectors (i.e. one-dimensional integer tensors) of the aforementioned integer types:

import type { Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new Int16Array([ 1, 2, 3, 4 ])
    }
});
const vector = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "vector": np.array([1, 2, 3, 4], dtype="int16")
    }
)
vector: np.ndarray = prediction.results[0]

import type { Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new Int16Array([ 1, 2, 3, 4 ])
    }
});
const vector = prediction.results[0] as Tensor;

import ai.muna.muna.types.Int16Tensor;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "vector" to shortArrayOf(1, 2, 3, 4)
    )
);
val vector = (Int16Tensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new () {
        ["vector"] = new short[] { 1, 2, 3, 4 }
    }
);
var vector = (Tensor<short>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("vector".to_string(), Value::Tensor(Tensor {
    data: TensorData::Int16(vec![1, 2, 3, 4]),
    shape: vec![4],
}));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(vector) = &prediction.results.unwrap()[0] else { unreachable!() };

Although Muna supports input vectors, compiled models will always output either scalars or Tensor instances—never plain vectors.

Muna supports integer tensors:

import type { Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/transpose",
    inputs: {
        matrix: {
            data: new Int16Array([ 1, 2, 3, 4 ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/transpose",
    inputs={
        "matrix": np.array([ [1, 2], [3, 4] ], dtype="int16")
    }
)
matrix: np.ndarray = prediction.results[0]

import type { Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/transpose",
    inputs: {
        matrix: {
            data: new Int16Array([ 1, 2, 3, 4 ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import ai.muna.muna.types.Int16Tensor;

val prediction = muna.predictions.create(
    "@fxn/transpose",
    mapOf(
        "matrix" to Int16Tensor(
            shortArrayOf(1, 2, 3, 4),
            intArrayOf(2, 2)
        )
    )
);
val matrix = (Int16Tensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/transpose",
    inputs: new() {
        ["matrix"] = new Tensor<short>(
            data: new [] { 1, 2, 3, 4 },
            shape: new [] { 2, 2 }
        )
    }
);
var radius = (Tensor<short>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("matrix".to_string(), Value::Tensor(Tensor {
    data: TensorData::Int16(vec![1, 2, 3, 4]),
    shape: vec![2, 2],
}));
let prediction = muna.predictions.create(
    "@fxn/transpose",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(matrix) = &prediction.results.unwrap()[0] else { unreachable!() };

Unsigned integer tensors are not supported in our Android client because of missing language support in Java.

Boolean Values

Muna supports boolean scalars:

const prediction = await muna.predictions.create({
    tag: "@fxn/negate",
    inputs: {
        value: true
    }
});
const truthy = prediction.results[0] as boolean;

prediction = muna.predictions.create(
    tag="@fxn/negate",
    inputs={
        "value": True
    }
)
truthy: bool = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/negate",
    inputs: {
        value: true
    }
});
const truthy = prediction.results[0] as boolean;

val prediction = muna.predictions.create(
    "@fxn/negate",
    mapOf(
        "value" to true
    )
);
val truthy = (Boolean)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/negate",
    inputs: new () {
        ["value"] = true
    }
);
var truthy = (bool)prediction.results[0];

let mut inputs = HashMap::new();
inputs.insert("value".to_string(), true.into());
let prediction = muna.predictions.create(
    "@fxn/negate",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Bool(truthy) = &prediction.results.unwrap()[0] else { unreachable!() };

Muna supports boolean vectors (i.e. one-dimensional boolean tensors):

import { BoolArray, type Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new BoolArray([ true, true, false, true ])
    }
});
const vector = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "vector": np.array([True, True, False, True], dtype="bool")
    }
)
vector: np.ndarray = prediction.results[0]

import { BoolArray, type Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new BoolArray([ true, true, false, true ])
    }
});
const vector = prediction.results[0] as Tensor;

import ai.muna.muna.types.BoolTensor;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "vector" to booleanArrayOf(true, true, false, true)
    )
);
val vector = (BoolTensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new () {
        ["vector"] = new[] { true, true, false, true }
    }
);
var vector = (Tensor<bool>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("vector".to_string(), Value::Tensor(Tensor {
    data: TensorData::Bool(vec![true, true, false, true]),
    shape: vec![4],
}));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(vector) = &prediction.results.unwrap()[0] else { unreachable!() };

Although Muna supports input vectors, compiled models will always output either scalars or Tensor instances—never plain vectors.

Muna supports boolean tensors:

import { BoolArray, type Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/transpose",
    inputs: {
        matrix: {
            data: new BoolArray([ true, true, false, true ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/transpose",
    inputs={
        "matrix": np.array([ [True, True], [False, True] ], dtype="bool")
    }
)
matrix: np.ndarray = prediction.results[0]

import { BoolArray, type Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/transpose",
    inputs: {
        matrix: {
            data: new BoolArray([ true, true, false, true ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import ai.muna.muna.types.BoolTensor;

val prediction = muna.predictions.create(
    "@fxn/transpose",
    mapOf(
        "matrix" to BoolTensor(
            booleanArrayOf(true, true, false, true),
            intArrayOf(2, 2)
        )
    )
);
val matrix = (BoolTensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/transpose",
    inputs: new () {
        ["matrix"] = new Tensor<bool>(
            data: new [] { true, true, false, true },
            shape: new [] { 2, 2 }
        )
    }
);
var radius = (Tensor<bool>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("matrix".to_string(), Value::Tensor(Tensor {
    data: TensorData::Bool(vec![true, true, false, true]),
    shape: vec![2, 2],
}));
let prediction = muna.predictions.create(
    "@fxn/transpose",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(matrix) = &prediction.results.unwrap()[0] else { unreachable!() };

Muna assumes that boolean values are 1 byte.

String Values

Muna supports string values:

const prediction = await muna.predictions.create({
    tag: "@fxn/upper",
    inputs: {
        text: "hello from function"
    }
});
const uppercase = prediction.results[0] as string;

prediction = muna.predictions.create(
    tag="@fxn/upper",
    inputs={
        "text": "hello from function"
    }
)
uppercase: str = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/upper",
    inputs: {
        text: "hello from function"
    }
});
const uppercase = prediction.results[0] as string;

val prediction = muna.predictions.create(
    "@fxn/upper",
    mapOf(
        "text" to "hello from function"
    )
);
val uppercase = (String)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/upper",
    inputs: new () {
        ["text"] = "hello from function"
    }
);
var uppercase = prediction.results[0] as string;

let mut inputs = HashMap::new();
inputs.insert("text".to_string(), "hello from function".into());
let prediction = muna.predictions.create(
    "@fxn/upper",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::String(uppercase) = &prediction.results.unwrap()[0] else { unreachable!() };

List Values

Muna supports lists of values, each with potentially different types:

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        elements: ["hello", 10, false]
    }
});
const elements = prediction.results[0] as any[];

from typing import Any, List

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "elements": ["hello", 10, False]
    }
)
elements: List[Any] = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        elements: ["hello", 10, false]
    }
});
const elements = prediction.results[0] as any[];

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "elements" to listOf("hello", 10, false)
    )
);
val elements = (List<Any>)prediction.results[0];

using Newtonsoft.Json.Linq;

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new() {
        ["elements"] = new object[] { "hello", 10, false } // can be any `T : IList`
    }
);
var elements = prediction.results[0] as JArray;

use muna::Value;

let mut inputs = HashMap::new();
inputs.insert("elements".to_string(), Value::List(vec![
    serde_json::json!("hello"),
    serde_json::json!(10),
    serde_json::json!(false),
]));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::List(elements) = &prediction.results.unwrap()[0] else { unreachable!() };

Input list values must be JSON-serializable.

Dictionary Values

Muna supports dictionary values:

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        person: {
            name: "Sara",
            age: 27
        }
    }
});
const person = prediction.results[0] as Record<string, any>;

from typing import Any, Dict

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "person": {
            "name": "Sara",
            "age": 27
        }
    }
)
person: Dict[str, Any] = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        person: {
            name: "Sara",
            age: 27
        }
    }
});
const person = prediction.results[0] as Record<string, any>;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "person" to mapOf(
            "name" to "Sara",
            "age" to 27
        )
    )
);
val person = (Map<String, Any>)prediction.results[0];

using System.Collections.Generic;
using Newtonsoft.Json.Linq;

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new() {
        ["person"] = new Dictionary<string, object> { // can be any `T : IDictionary`
            ["name"] = "Sara",
            ["age"] = 27
        }
    }
);
var person = prediction.results[0] as JObject;

use muna::Value;

let mut inputs = HashMap::new();
let mut person = serde_json::Map::new();
person.insert("name".to_string(), serde_json::json!("Sara"));
person.insert("age".to_string(), serde_json::json!(27));
inputs.insert("person".to_string(), Value::Dict(person));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Dict(person) = &prediction.results.unwrap()[0] else { unreachable!() };

Input dictionary values must be JSON-serializable.

Image Values

Muna supports images, represented as raw pixel buffers with 8 bytes per pixel and interleaved by channel. Muna supports three pixel buffer formats:

Pixel format	Channels	Description
`A8`	1	Single channel luminance or alpha image.
`RGB888`	3	Color image without alpha channel.
`RGBA8888`	4	Color image with alpha channel.

Some client SDKs provide Image utility types for working with images:

import type { Image } from "muna"

const prediction = await muna.predictions.create({
    tag: "@vision-co/remove-background",
    inputs: {
        image: {
            data: new Uint8ClampedArray(1280 * 720 * 3),
            width: 1280,
            height: 720,
            channels: 3
        } satisfies Image
    }
});
const image = prediction.results[0] as Image;

from PIL import Image

prediction = muna.predictions.create(
    tag="@vision-co/remove-background",
    inputs={
        "image": Image.open("cat.jpg")
    }
)
image: Image.Image = prediction.results[0]

import type { Image } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@vision-co/remove-background",
    inputs: {
        image: {
            data: new Uint8ClampedArray(1280 * 720 * 3),
            width: 1280,
            height: 720,
            channels: 3
        } satisfies Image
    }
});
const image = prediction.results[0] as Image;

import ai.muna.muna.types.Image;

val prediction = muna.predictions.create(
    "@vision-co/remove-background",
    mapOf(
        "image" to Image(
            ByteBuffer.allocateDirect(1280 * 720 * 3),
            1280, // width
            720,  // height
            3     // channels
        )
    )
);
val image = (Image)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@vision-co/remove-background",
    inputs: new () {
        ["image"] = new Image(
            data: new byte[1280 * 720 * 3],
            width: 1280,
            height: 720,
            channels: 3
        )
    }
);
var image = (Image)prediction.results[0];

use muna::{Value, Image};

let mut inputs = HashMap::new();
inputs.insert("image".to_string(), Value::Image(Image {
    data: vec![0u8; 1280 * 720 * 3],
    width: 1280,
    height: 720,
    channels: 3,
}));
let prediction = muna.predictions.create(
    "@vision-co/remove-background",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Image(image) = &prediction.results.unwrap()[0] else { unreachable!() };

Binary Values

Muna supports binary blobs:

const prediction = await muna.predictions.create({
    tag: "@vision-co/decode-jpeg",
    inputs: {
        buffer: new ArrayBuffer(1024)
    }
});
const buffer = prediction.results[0] as ArrayBuffer;

from io import BytesIO

prediction = muna.predictions.create(
    tag="@vision-co/decode-jpeg",
    inputs={
        "buffer": BytesIO(b"\x00\x01") # or `bytes`, `bytearray`, `memoryview`
    }
)
buffer: BytesIO = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@vision-co/decode-jpeg",
    inputs: {
        buffer: new ArrayBuffer(1024)
    }
});
const buffer = prediction.results[0] as ArrayBuffer;

val prediction = muna.predictions.create(
    "@vision-co/decode-png",
    mapOf(
        "buffer" to ByteArrayInputStream(byteArrayOf(0x1, 0x2)) // must be an `InputStream`
    )
);
val buffer = (InputStream)prediction.results[0];

using System.IO;

var prediction = await muna.Predictions.Create(
    tag: "@vision-co/decode-jpeg",
    inputs: new () {
        ["buffer"] = new MemoryStream() // must be a `Stream`
    }
);
var buffer = (Stream)prediction.results[0];

use muna::Value;

let mut inputs = HashMap::new();
inputs.insert("buffer".to_string(), Value::Binary(vec![0x00, 0x01]));
let prediction = muna.predictions.create(
    "@vision-co/decode-jpeg",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Binary(buffer) = &prediction.results.unwrap()[0] else { unreachable!() };

Because Muna’s security model prohibits file system access, binary input values are always fully read into memory before being passed to the compiled model.To run inference on large files, consider mapping the file into memory using mmap or your environment’s equivalent.

Specifying Inference Acceleration

Muna lets you choose which processor runs inference on the local device, per-request. Specify an acceleration when creating or streaming a prediction:

// 🔥 Run inference with the local NPU
const prediction = await muna.predictions.create({
  tag: "@bytedance/depth-anything-3",
  inputs: { image },
  acceleration: "local_npu"
});

# 🔥 Run inference with the local NPU
prediction = muna.predictions.create(
  tag="@bytedance/depth-anything-3",
  inputs={ "image": image },
  acceleration="local_npu"
)

// 🔥 Run inference with the local NPU
const prediction = await muna.predictions.create({
  tag: "@bytedance/depth-anything-3",
  inputs: { image },
  acceleration: "local_npu"
});

// 🔥 Run inference with the Apple Neural Engine
let prediction = try await muna.predictions.create(
  tag: "@bytedance/depth-anything-3",
  inputs: ["image": image],
  acceleration: .npu
)

// 🔥 Run inference with the local NPU
val prediction = muna.predictions.create(
  "@bytedance/depth-anything-3",
  mapOf("image" to image),
  Acceleration.NPU
)

// 🔥 Run inference with the local NPU
var prediction = await muna.Predictions.Create(
  tag: "@bytedance/depth-anything-3",
  inputs: new() { ["image"] = image },
  acceleration: @"local_npu"
);

Specifying the Acceleration

Below are the currently supported acceleration specifiers:

Acceleration	Notes
`local_cpu`	Use the CPU to accelerate inference.
`local_gpu`	Use the GPU to accelerate inference.
`local_npu`	Use the neural processor to accelerate inference.

Specifying the Device

Some Muna clients allow you to specify the acceleration device used to run inference. Our clients expose this field as an untyped integer or pointer. The underlying type depends on the current operating system:

Android

Currently unsupported.

iOS

The device is reinterpreted as a Metal device with type id<MTLDevice>.

Linux

The device is reinterpeted as a pointer to a CUDA device ID or HIP device ID with type int*.

macOS

The device is reinterpreted as a Metal device with type id<MTLDevice>.

Web

The device is reinterpreted as a WebGPU device with type GPUDevice.

Windows

The device is reinterpreted as a DirectX 12 device with type ID3D12Device*.

You should never specify the inference device unless you know what the hell you’re doing.

The inference device is merely a hint. Setting a device does not guarantee that all or any operation in the compiled function will actually use that acceleration device.

Get Started

Coding Agents

Local Inference

Using the Muna Client

Creating Predictions

Streaming Predictions

Consuming Inference Streams

Using Inference Values

Specifying Inference Acceleration

Specifying the Acceleration

Specifying the Device

​Creating Predictions

​Streaming Predictions

​Consuming Inference Streams

​Using Inference Values

​Specifying Inference Acceleration

​Specifying the Acceleration

​Specifying the Device

Creating Predictions

Streaming Predictions

Consuming Inference Streams

Using Inference Values

Specifying Inference Acceleration

Specifying the Acceleration

Specifying the Device