Skip to main content
Muna supports consuming the partial results of a prediction as they are made available by the predictor.

​
Consuming a Prediction Stream

Use the muna.predictions.stream function to consume a prediction stream: 
import { Muna } from "muna"

// 💥 Create a Muna client
const muna = new Muna({ accessKey: "..." });

// 🔥 Create a prediction stream
const stream = await muna.predictions.stream({
    tag: "@text-co/split-sentence",
    inputs: { text: "Hello world" }
});

// 🚀 Consume the stream
for await (const prediction of stream) {
    console.log(prediction.results[0]);
}
You can use prediction streaming to implement text generation user interfaces for LLMs.

​
Creating vs. Streaming Predictions

Streaming in Muna is designed to be highly intuitive to use. We fully separate how a prediction function is implemented (i.e. eager vs generator functions) from how the function might be consumed (i.e. creating vs. streaming). Consider these two predictors: 
def predict():
    return "hello from Muna"
You can choose how to consume a prediction function depending on what works best for your user experience.
Here are the reuslts of creating vs. streaming each function at runtime:
In this case, the single prediction is returned:
// Create a prediction with the eager predictor
const prediction = await muna.predictions.create({
  tag: "@muna/eager",
  inputs: { }
});

// Display the results
console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"
In this case, the Muna client will consume all partial predictions yielded by the predictor then return the very last one:
// Create a prediction with the streaming predictor
const prediction = await muna.predictions.create({
  tag: "@muna/generator",
  inputs: { }
});

// Display the results
console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"
In this case, the Muna client will return a prediction stream with the single prediction returned by the predictor:
// Create a prediction stream with the eager predictor
const stream = await muna.predictions.stream({
  tag: "@muna/eager",
  inputs: { }
});

// Display the results
for await (const prediction of stream)
  console.log(prediction.results[0]);

// Outputs:
// "hello from Muna"
In this case, the Muna client will provide a prediction stream containing all partial predictions yielded by the predictor:
// Create a prediction stream with the generator predictor
const stream = await muna.predictions.stream({
  tag: "@muna/generator",
  inputs: { }
});

// Display the results
for await (const prediction of stream)
  console.log(prediction.results[0]);

// Outputs:
// "hello"
// "hello from"
// "hello from Muna"