Skip to main content
Muna supports consuming the partial results of a prediction as they are made available by the predictor.

​
Consuming a Prediction Stream

Use the muna.predictions.stream function to consume a prediction stream: 
import { Muna } from "muna"

// 💥 Create a Muna client
const muna = new Muna({ accessKey: "..." });

// 🔥 Create a prediction stream
const stream = await muna.predictions.stream({
    tag: "@text-co/split-sentence",
    inputs: { text: "Hello world" }
});

// 🚀 Consume the stream
for await (const prediction of stream) {
    console.log(prediction.results[0]);
}
You can use prediction streaming to implement text generation user interfaces for LLMs.

​
Creating vs. Streaming Predictions

Streaming in Muna is designed to be highly intuitive to use. We fully separate how a prediction function is implemented (i.e. eager vs generator functions) from how the function might be consumed (i.e. creating vs. streaming). Consider these two predictors: 
def predict():
    return "hello from Muna"
You can choose how to consume a prediction function depending on what works best for your user experience. You don’t have to care about the underlying function!
Here are the reuslts of creating vs. streaming each function at runtime:
In this case, the single prediction is returned:
// Create a prediction with the eager predictor
const prediction = await muna.predictions.create({
  tag: "@muna/eager",
  inputs: { }
});

// Display the results
console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"
In this case, the Muna client will consume all partial predictions yielded by the predictor then return the very last one:
// Create a prediction with the streaming predictor
const prediction = await muna.predictions.create({
  tag: "@muna/generator",
  inputs: { }
});

// Display the results
console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"
In this case, the Muna client will return a prediction stream with the single prediction returned by the predictor:
// Create a prediction stream with the eager predictor
const stream = await muna.predictions.stream({
  tag: "@muna/eager",
  inputs: { }
});

// Display the results
for await (const prediction of stream)
  console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"
In this case, the Muna client will provide a prediction stream containing all partial predictions yielded by the predictor:
// Create a prediction stream with the generator predictor
const stream = await muna.predictions.stream({
  tag: "@muna/generator",
  inputs: { }
});

// Display the results
for await (const prediction of stream)
  console.log(prediction.results[0]);

// Outputs:
// "hello"
// "hello from"
// "hello from Muna"