Running Inference

The very first step in making predictions is finding or compiling a model:

Explore Models

Explore public models on Muna.

Compile a Model

Compile a model with Muna.

Making Predictions

Making predictions with Muna can be done in as little as two lines of code.

import { Muna } from "muna"

// 💥 Create your Muna client
const muna = new Muna({ accessKey: "..." });

// 🔥 Run the prediction locally
const prediction = await muna.predictions.create({
    tag: "@fxn/greeting",
    inputs: { name: "Yusuf" }
});

// 🚀 Print the result
console.log(prediction.results[0]);

from muna import Muna

# 💥 Create your Muna client
muna = Muna(access_key="...")

# 🔥 Run the prediction locally
prediction = muna.predictions.create(
    tag="@fxn/greeting",
    inputs={ "name": "Muna" }
)

# 🚀 Use the results
print(prediction.results[0])

import { Muna } from "@muna/expo"

// 💥 Create your Muna client
const muna = new Muna({ accessKey: "..." });

// 🔥 Run the prediction locally
const prediction = await muna.predictions.create({
    tag: "@fxn/greeting",
    inputs: { name: "Yusuf" }
});

// 🚀 Print the result
console.log(prediction.results[0]);

import Muna

// 💥 Create your Muna client
let muna = Muna(accessKey: "...")

// 🔥 Run the prediction locally
let prediction = try await muna.predictions.create(
    tag: "@fxn/greeting",
    inputs: [ "name": "Terri" ]
)

// 🚀 Use the results
print("\(prediction.results![0]!)")

import ai.muna.muna.Muna

// 💥 Create your Muna client
val muna = Muna("...")

// 🔥 Run the prediction locally
val prediction = muna.predictions.create(
    "@fxn/greeting",
    mapOf("name" to "Timi")
)

// 🚀 Use the results
println(prediction.results!![0])

using Muna;

// 💥 Create your Muna client
var muna = new Muna(accessKey: "...");

// 🔥 Run the prediction locally
var prediction = await muna.Predictions.Create(
    tag: "@fxn/greeting",
    inputs: new() { ["name"] = "Peter" }
);

// 🚀 Use the results
Debug.Log(prediction.results[0]);

use std::collections::HashMap;
use muna::{Muna, Value};

// 💥 Create your Muna client
let muna = Muna::new(Some("..."), None);

// 🔥 Run the prediction locally
let mut inputs = HashMap::new();
inputs.insert("name".to_string(), "Yusuf".into());
let prediction = muna.predictions.create(
    "@fxn/greeting",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();

// 🚀 Print the result
println!("{:?}", prediction.results.unwrap()[0]);

Using Prediction Values

Muna supports a fixed set of value types for prediction input and output values:

Floating Point Values

Muna supports the following floating-point numbers:

Muna value type	C/C++ type	Description
`float16`	`float16_t`	IEEE 754 16-bit floating point number.
`float32`	`float`	IEEE 754 32-bit floating point number.
`float64`	`double`	IEEE 754 64-bit floating point number.

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        radius: 4.5
    }
});
const radius = prediction.results[0] as number;

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "radius": 4.5
    }
)
radius: float = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        radius: 4.5
    }
});
const radius = prediction.results[0] as number;

let prediction = try await muna.predictions.create(
    tag: "@fxn/identity",
    inputs: [
        "radius": 4.5
    ]
)
let radius = prediction.results![0] as! Float

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "radius" to 4.5f
    )
);
val radius = (Float)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new() {
        ["radius"] = 4.5f
    }
);
var radius = (float)prediction.results[0];

let mut inputs = HashMap::new();
inputs.insert("radius".to_string(), 4.5f32.into());
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Float(radius) = &prediction.results.unwrap()[0] else { unreachable!() };

In languages that don’t support fixed-size floating point scalars, the data type for floating point values defaults to float32. Use a tensor constructor to explicitly specify the data type.

Support for half-precision floating point scalars float16 is planned for the future depending on language support.

Muna supports floating point vectors (i.e. one-dimensional floating point tensors):

import type { Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new Float32Array([ 1.2, 2.2, 3.2, 4.5 ])
    }
});
const vector = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "vector": np.array([1.2, 2.2, 3.2, 4.5], dtype="float32")
    }
)
vector: np.ndarray = prediction.results[0]

import type { Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new Float32Array([ 1.2, 2.2, 3.2, 4.5 ])
    }
});
const vector = prediction.results[0] as Tensor;

import ai.muna.muna.types.Float32Tensor;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "vector" to floatArrayOf(1.2f, 2.2f, 3.2f, 4.5f)
    )
);
val vector = (Float32Tensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new() {
        ["vector"] = new [] { 1.2f, 2.2f, 3.2f, 4.5f }
    }
);
var vector = (Tensor<float>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("vector".to_string(), Value::Tensor(Tensor {
    data: TensorData::Float32(vec![1.2, 2.2, 3.2, 4.5]),
    shape: vec![4],
}));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(vector) = &prediction.results.unwrap()[0] else { unreachable!() };

Although Muna supports input vectors, predictors will always output either scalars or Tensor instances—never plain vectors.

Muna supports floating point tensors:

import type { Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        matrix: {
            data: new Float64Array([ 1.2, 2.2, 3.2, 4.5 ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "matrix": np.array([ [1.2, 2.2], [3.2, 4.5] ], dtype="float64")
    }
)
matrix: np.ndarray = prediction.results[0]

import type { Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        matrix: {
            data: new Float64Array([ 1.2, 2.2, 3.2, 4.5 ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import ai.muna.muna.types.Float64Tensor;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "matrix" to Float64Tensor(
            doubleArrayOf(1.2, 2.2, 3.2, 4.5), // data
            intArrayOf(2, 2) // shape
        )
    )
);
val matrix = (Float64Tensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new () {
        ["matrix"] = new Tensor<double>(
            data: new [] { 1.2, 2.2, 3.2, 4.5 },
            shape: new [] { 2, 2 }
        )
    }
);
var radius = (Tensor<double>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("matrix".to_string(), Value::Tensor(Tensor {
    data: TensorData::Float64(vec![1.2, 2.2, 3.2, 4.5]),
    shape: vec![2, 2],
}));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(matrix) = &prediction.results.unwrap()[0] else { unreachable!() };

Integer Values

Muna supports several signed and unsigned integer scalars:

Muna value type	C/C++ type	Description
`int8`	`int8_t`	Signed 8-bit integer.
`int16`	`int16_t`	Signed 16-bit integer.
`int32`	`int32_t`	Signed 32-bit integer.
`int64`	`int64_t`	Signed 64-bit integer.
`uint8`	`uint8_t`	Unsigned 8-bit integer.
`uint16`	`uint16_t`	Unsigned 16-bit integer.
`uint32`	`uint32_t`	Unsigned 32-bit integer.
`uint64`	`uint64_t`	Unsigned 64-bit integer.

const prediction = await muna.predictions.create({
    tag: "@fxn/squeeze",
    inputs: {
        oranges: 12
    }
});
const cups = prediction.results[0] as number;

prediction = muna.predictions.create(
    tag="@fxn/squeeze",
    inputs={
        "oranges": 12
    }
)
cups: int = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/squeeze",
    inputs: {
        oranges: 12
    }
});
const cups = prediction.results[0] as number;

val prediction = muna.predictions.create(
    "@fxn/squeeze",
    mapOf(
        "oranges" to 12
    )
);
val cups = (Int)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/squeeze",
    inputs: new() {
        ["oranges"] = 12
    }
);
var cups = (int)prediction.results[0];

let mut inputs = HashMap::new();
inputs.insert("oranges".to_string(), 12i32.into());
let prediction = muna.predictions.create(
    "@fxn/squeeze",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Int(cups) = &prediction.results.unwrap()[0] else { unreachable!() };

When integer scalars are passed to predictors, the data type defaults to int32. Use a tensor constructor to explicitly specify the data type.

Muna supports integer vectors (i.e. one-dimensional integer tensors) of the aforementioned integer types:

import type { Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new Int16Array([ 1, 2, 3, 4 ])
    }
});
const vector = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "vector": np.array([1, 2, 3, 4], dtype="int16")
    }
)
vector: np.ndarray = prediction.results[0]

import type { Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new Int16Array([ 1, 2, 3, 4 ])
    }
});
const vector = prediction.results[0] as Tensor;

import ai.muna.muna.types.Int16Tensor;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "vector" to shortArrayOf(1, 2, 3, 4)
    )
);
val vector = (Int16Tensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new () {
        ["vector"] = new short[] { 1, 2, 3, 4 }
    }
);
var vector = (Tensor<short>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("vector".to_string(), Value::Tensor(Tensor {
    data: TensorData::Int16(vec![1, 2, 3, 4]),
    shape: vec![4],
}));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(vector) = &prediction.results.unwrap()[0] else { unreachable!() };

Although Muna supports input vectors, predictors will always output either scalars or Tensor instances—never plain vectors.

Muna supports integer tensors:

import type { Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/transpose",
    inputs: {
        matrix: {
            data: new Int16Array([ 1, 2, 3, 4 ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/transpose",
    inputs={
        "matrix": np.array([ [1, 2], [3, 4] ], dtype="int16")
    }
)
matrix: np.ndarray = prediction.results[0]

import type { Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/transpose",
    inputs: {
        matrix: {
            data: new Int16Array([ 1, 2, 3, 4 ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import ai.muna.muna.types.Int16Tensor;

val prediction = muna.predictions.create(
    "@fxn/transpose",
    mapOf(
        "matrix" to Int16Tensor(
            shortArrayOf(1, 2, 3, 4),
            intArrayOf(2, 2)
        )
    )
);
val matrix = (Int16Tensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/transpose",
    inputs: new() {
        ["matrix"] = new Tensor<short>(
            data: new [] { 1, 2, 3, 4 },
            shape: new [] { 2, 2 }
        )
    }
);
var radius = (Tensor<short>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("matrix".to_string(), Value::Tensor(Tensor {
    data: TensorData::Int16(vec![1, 2, 3, 4]),
    shape: vec![2, 2],
}));
let prediction = muna.predictions.create(
    "@fxn/transpose",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(matrix) = &prediction.results.unwrap()[0] else { unreachable!() };

Unsigned integer tensors are not supported in our Android client because of missing language support in Java.

Boolean Values

Muna supports boolean scalars:

const prediction = await muna.predictions.create({
    tag: "@fxn/negate",
    inputs: {
        value: true
    }
});
const truthy = prediction.results[0] as boolean;

prediction = muna.predictions.create(
    tag="@fxn/negate",
    inputs={
        "value": True
    }
)
truthy: bool = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/negate",
    inputs: {
        value: true
    }
});
const truthy = prediction.results[0] as boolean;

val prediction = muna.predictions.create(
    "@fxn/negate",
    mapOf(
        "value" to true
    )
);
val truthy = (Boolean)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/negate",
    inputs: new () {
        ["value"] = true
    }
);
var truthy = (bool)prediction.results[0];

let mut inputs = HashMap::new();
inputs.insert("value".to_string(), true.into());
let prediction = muna.predictions.create(
    "@fxn/negate",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Bool(truthy) = &prediction.results.unwrap()[0] else { unreachable!() };

Muna supports boolean vectors (i.e. one-dimensional boolean tensors):

import { BoolArray, type Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new BoolArray([ true, true, false, true ])
    }
});
const vector = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "vector": np.array([True, True, False, True], dtype="bool")
    }
)
vector: np.ndarray = prediction.results[0]

import { BoolArray, type Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        vector: new BoolArray([ true, true, false, true ])
    }
});
const vector = prediction.results[0] as Tensor;

import ai.muna.muna.types.BoolTensor;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "vector" to booleanArrayOf(true, true, false, true)
    )
);
val vector = (BoolTensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new () {
        ["vector"] = new[] { true, true, false, true }
    }
);
var vector = (Tensor<bool>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("vector".to_string(), Value::Tensor(Tensor {
    data: TensorData::Bool(vec![true, true, false, true]),
    shape: vec![4],
}));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(vector) = &prediction.results.unwrap()[0] else { unreachable!() };

Although Muna supports input vectors, predictors will always output either scalars or Tensor instances—never plain vectors.

Muna supports boolean tensors:

import { BoolArray, type Tensor } from "muna"

const prediction = await muna.predictions.create({
    tag: "@fxn/transpose",
    inputs: {
        matrix: {
            data: new BoolArray([ true, true, false, true ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import numpy as np

prediction = muna.predictions.create(
    tag="@fxn/transpose",
    inputs={
        "matrix": np.array([ [True, True], [False, True] ], dtype="bool")
    }
)
matrix: np.ndarray = prediction.results[0]

import { BoolArray, type Tensor } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@fxn/transpose",
    inputs: {
        matrix: {
            data: new BoolArray([ true, true, false, true ]),
            shape: [2, 2]
        } satisfies Tensor
    }
});
const matrix = prediction.results[0] as Tensor;

import ai.muna.muna.types.BoolTensor;

val prediction = muna.predictions.create(
    "@fxn/transpose",
    mapOf(
        "matrix" to BoolTensor(
            booleanArrayOf(true, true, false, true),
            intArrayOf(2, 2)
        )
    )
);
val matrix = (BoolTensor)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/transpose",
    inputs: new () {
        ["matrix"] = new Tensor<bool>(
            data: new [] { true, true, false, true },
            shape: new [] { 2, 2 }
        )
    }
);
var radius = (Tensor<bool>)prediction.results[0];

use muna::{Value, Tensor, TensorData};

let mut inputs = HashMap::new();
inputs.insert("matrix".to_string(), Value::Tensor(Tensor {
    data: TensorData::Bool(vec![true, true, false, true]),
    shape: vec![2, 2],
}));
let prediction = muna.predictions.create(
    "@fxn/transpose",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Tensor(matrix) = &prediction.results.unwrap()[0] else { unreachable!() };

Muna assumes that boolean values are 1 byte.

String Values

Muna supports string values:

const prediction = await muna.predictions.create({
    tag: "@fxn/upper",
    inputs: {
        text: "hello from function"
    }
});
const uppercase = prediction.results[0] as string;

prediction = muna.predictions.create(
    tag="@fxn/upper",
    inputs={
        "text": "hello from function"
    }
)
uppercase: str = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/upper",
    inputs: {
        text: "hello from function"
    }
});
const uppercase = prediction.results[0] as string;

val prediction = muna.predictions.create(
    "@fxn/upper",
    mapOf(
        "text" to "hello from function"
    )
);
val uppercase = (String)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@fxn/upper",
    inputs: new () {
        ["text"] = "hello from function"
    }
);
var uppercase = prediction.results[0] as string;

let mut inputs = HashMap::new();
inputs.insert("text".to_string(), "hello from function".into());
let prediction = muna.predictions.create(
    "@fxn/upper",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::String(uppercase) = &prediction.results.unwrap()[0] else { unreachable!() };

List Values

Muna supports lists of values, each with potentially different types:

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        elements: ["hello", 10, false]
    }
});
const elements = prediction.results[0] as any[];

from typing import Any, List

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "elements": ["hello", 10, False]
    }
)
elements: List[Any] = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        elements: ["hello", 10, false]
    }
});
const elements = prediction.results[0] as any[];

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "elements" to listOf("hello", 10, false)
    )
);
val elements = (List<Any>)prediction.results[0];

using Newtonsoft.Json.Linq;

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new() {
        ["elements"] = new object[] { "hello", 10, false } // can be any `T : IList`
    }
);
var elements = prediction.results[0] as JArray;

use muna::Value;

let mut inputs = HashMap::new();
inputs.insert("elements".to_string(), Value::List(vec![
    serde_json::json!("hello"),
    serde_json::json!(10),
    serde_json::json!(false),
]));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::List(elements) = &prediction.results.unwrap()[0] else { unreachable!() };

Input list values must be JSON-serializable.

Dictionary Values

Muna supports dictionary values:

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        person: {
            name: "Sara",
            age: 27
        }
    }
});
const person = prediction.results[0] as Record<string, any>;

from typing import Any, Dict

prediction = muna.predictions.create(
    tag="@fxn/identity",
    inputs={
        "person": {
            "name": "Sara",
            "age": 27
        }
    }
)
person: Dict[str, Any] = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@fxn/identity",
    inputs: {
        person: {
            name: "Sara",
            age: 27
        }
    }
});
const person = prediction.results[0] as Record<string, any>;

val prediction = muna.predictions.create(
    "@fxn/identity",
    mapOf(
        "person" to mapOf(
            "name" to "Sara",
            "age" to 27
        )
    )
);
val person = (Map<String, Any>)prediction.results[0];

using System.Collections.Generic;
using Newtonsoft.Json.Linq;

var prediction = await muna.Predictions.Create(
    tag: "@fxn/identity",
    inputs: new() {
        ["person"] = new Dictionary<string, object> { // can be any `T : IDictionary`
            ["name"] = "Sara",
            ["age"] = 27
        }
    }
);
var person = prediction.results[0] as JObject;

use muna::Value;

let mut inputs = HashMap::new();
let mut person = serde_json::Map::new();
person.insert("name".to_string(), serde_json::json!("Sara"));
person.insert("age".to_string(), serde_json::json!(27));
inputs.insert("person".to_string(), Value::Dict(person));
let prediction = muna.predictions.create(
    "@fxn/identity",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Dict(person) = &prediction.results.unwrap()[0] else { unreachable!() };

Input dictionary values must be JSON-serializable.

Image Values

Muna supports images, represented as raw pixel buffers with 8 bytes per pixel and interleaved by channel. Muna supports three pixel buffer formats:

Pixel format	Channels	Description
`A8`	1	Single channel luminance or alpha image.
`RGB888`	3	Color image without alpha channel.
`RGBA8888`	4	Color image with alpha channel.

Some client SDKs provide Image utility types for working with images:

import type { Image } from "muna"

const prediction = await muna.predictions.create({
    tag: "@vision-co/remove-background",
    inputs: {
        image: {
            data: new Uint8ClampedArray(1280 * 720 * 3),
            width: 1280,
            height: 720,
            channels: 3
        } satisfies Image
    }
});
const image = prediction.results[0] as Image;

from PIL import Image

prediction = muna.predictions.create(
    tag="@vision-co/remove-background",
    inputs={
        "image": Image.open("cat.jpg")
    }
)
image: Image.Image = prediction.results[0]

import type { Image } from "@muna/expo"

const prediction = await muna.predictions.create({
    tag: "@vision-co/remove-background",
    inputs: {
        image: {
            data: new Uint8ClampedArray(1280 * 720 * 3),
            width: 1280,
            height: 720,
            channels: 3
        } satisfies Image
    }
});
const image = prediction.results[0] as Image;

import ai.muna.muna.types.Image;

val prediction = muna.predictions.create(
    "@vision-co/remove-background",
    mapOf(
        "image" to Image(
            ByteBuffer.allocateDirect(1280 * 720 * 3),
            1280, // width
            720,  // height
            3     // channels
        )
    )
);
val image = (Image)prediction.results[0];

var prediction = await muna.Predictions.Create(
    tag: "@vision-co/remove-background",
    inputs: new () {
        ["image"] = new Image(
            data: new byte[1280 * 720 * 3],
            width: 1280,
            height: 720,
            channels: 3
        )
    }
);
var image = (Image)prediction.results[0];

use muna::{Value, Image};

let mut inputs = HashMap::new();
inputs.insert("image".to_string(), Value::Image(Image {
    data: vec![0u8; 1280 * 720 * 3],
    width: 1280,
    height: 720,
    channels: 3,
}));
let prediction = muna.predictions.create(
    "@vision-co/remove-background",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Image(image) = &prediction.results.unwrap()[0] else { unreachable!() };

Binary Values

Muna supports binary blobs:

const prediction = await muna.predictions.create({
    tag: "@vision-co/decode-jpeg",
    inputs: {
        buffer: new ArrayBuffer(1024)
    }
});
const buffer = prediction.results[0] as ArrayBuffer;

from io import BytesIO

prediction = muna.predictions.create(
    tag="@vision-co/decode-jpeg",
    inputs={
        "buffer": BytesIO(b"\x00\x01") # or `bytes`, `bytearray`, `memoryview`
    }
)
buffer: BytesIO = prediction.results[0]

const prediction = await muna.predictions.create({
    tag: "@vision-co/decode-jpeg",
    inputs: {
        buffer: new ArrayBuffer(1024)
    }
});
const buffer = prediction.results[0] as ArrayBuffer;

val prediction = muna.predictions.create(
    "@vision-co/decode-png",
    mapOf(
        "buffer" to ByteArrayInputStream(byteArrayOf(0x1, 0x2)) // must be an `InputStream`
    )
);
val buffer = (InputStream)prediction.results[0];

using System.IO;

var prediction = await muna.Predictions.Create(
    tag: "@vision-co/decode-jpeg",
    inputs: new () {
        ["buffer"] = new MemoryStream() // must be a `Stream`
    }
);
var buffer = (Stream)prediction.results[0];

use muna::Value;

let mut inputs = HashMap::new();
inputs.insert("buffer".to_string(), Value::Binary(vec![0x00, 0x01]));
let prediction = muna.predictions.create(
    "@vision-co/decode-jpeg",
    Some(inputs),
    None,
    None,
    None,
).await.unwrap();
let Value::Binary(buffer) = &prediction.results.unwrap()[0] else { unreachable!() };

Because Muna’s security model prohibits file system access, binary input values are always fully read into memory before being passed to the predictor.To make predictions on large files, consider mapping the file into memory using mmap or your environment’s equivalent.

Streaming Predictions

Muna supports consuming the partial results of a prediction as they are made available by the predictor:

// 🔥 Create a prediction stream
const stream = await muna.predictions.stream({
  tag: "@text-co/split-sentence",
  inputs: { text: "Hello world" }
});

// 🚀 Consume the stream
for await (const prediction of stream)
  console.log(prediction.results[0]);

# 🔥 Create a prediction stream
stream = muna.predictions.stream(
  tag="@text-co/split-sentence",
  inputs={ "text": "Hello world" }
)

# 🚀 Consume the stream
for prediction in stream:
  print(prediction.results[0])

// 🔥 Create a prediction stream
const stream = await muna.predictions.stream({
  tag: "@text-co/split-sentence",
  inputs: { text: "Hello world" }
});

// 🚀 Consume the stream
for await (const prediction of stream)
  console.log(prediction.results[0]);

// 🔥 Create a prediction stream
let stream = try await muna.predictions.stream(
  tag: "@text-co/split-sentence",
  inputs: ["text": "Hello world"],
)

// 🚀 Consume the stream
for try await prediction in stream {
  print("\(prediction.results?[0])")
}

// 🔥 Create a prediction stream
val stream = muna.predictions.stream(
  "@text-co/split-sentence",
  mapOf("text" to "Hello world")
)

// 🚀 Consume the stream
stream.use {
  for (prediction in it.consume())
    println(prediction.results!![0])
}

// 🔥 Create a prediction stream
var stream = await muna.Predictions.Stream(
  tag: "@text-co/split-sentence",
  inputs: new() { ["text"] = "Hello world" }
);

// 🚀 Consume the stream
await foreach (var prediction in stream)
  Debug.Log(prediction.results[0]);

use std::collections::HashMap;
use futures_util::StreamExt;

// 🔥 Create a prediction stream
let mut inputs = HashMap::new();
inputs.insert("text".to_string(), "Hello world".into());
let mut stream = muna.predictions.stream(
  "@text-co/split-sentence",
  inputs,
  None,
).await.unwrap();

// 🚀 Consume the stream
while let Some(Ok(prediction)) = stream.next().await {
  println!("{:?}", prediction.results.unwrap()[0]);
}

Consuming Prediction Streams

Streaming in Muna is designed to fully separate how a prediction function is implemented from how the function might be consumed. Consider these two predictors:

def predict() -> str:
    return "hello from Muna"

def predict() -> Iterator[str]:
    yield "hello"
    yield "hello from"
    yield "hello from Muna"

Here are the reuslts of creating vs. streaming each function at runtime:

Creating Predictions with eager.py

In this case, the single prediction is returned:

// Create a prediction with the eager predictor
const prediction = await muna.predictions.create({
  tag: "@muna/eager",
  inputs: { }
});

// Display the results
console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"

Creating Predictions with generator.py

In this case, the Muna client will consume all partial predictions yielded by the predictor then return the very last one:

// Create a prediction with the streaming predictor
const prediction = await muna.predictions.create({
  tag: "@muna/generator",
  inputs: { }
});

// Display the results
console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"

Streaming Predictions with eager.py

In this case, the Muna client will return a prediction stream with the single prediction returned by the predictor:

// Create a prediction stream with the eager predictor
const stream = await muna.predictions.stream({
  tag: "@muna/eager",
  inputs: { }
});

// Display the results
for await (const prediction of stream)
  console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"

Streaming Predictions with generator.py

In this case, the Muna client will provide a prediction stream containing all partial predictions yielded by the predictor:

// Create a prediction stream with the generator predictor
const stream = await muna.predictions.stream({
  tag: "@muna/generator",
  inputs: { }
});

// Display the results
for await (const prediction of stream)
  console.log(prediction.results[0]);

// Outputs:
// "hello"
// "hello from"
// "hello from Muna"

You can choose how to consume a prediction function depending on what works best for your user experience. You don’t have to care about the underlying function!

Get Started

Running Models

Deploying Models

Insiders

Explore Models

Compile a Model

Making Predictions

Using Prediction Values

Streaming Predictions

Consuming Prediction Streams

Explore Models

Compile a Model

​Making Predictions

​Using Prediction Values

​Streaming Predictions

​Consuming Prediction Streams

Making Predictions

Using Prediction Values

Streaming Predictions

Consuming Prediction Streams