Streaming

For real-time output, set stream: true. Instead of waiting for the full response, you receive chunks as they're generated.

Basic Streaming

const stream = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Write a haiku about code" }],
  stream: true,
});

for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content ?? "";
  process.stdout.write(text);
}

Stream Object

When stream: true, the SDK returns a Stream<ChatCompletionChunk> object that implements AsyncIterable. Each chunk contains:

interface ChatCompletionChunk {
  id: string;
  object: string;
  created: number;
  model: string;
  choices: ChatCompletionChunkChoice[];
  usage?: CompletionUsage | null;
}

interface ChatCompletionChunkChoice {
  index: number;
  delta: { role?: string; content?: string | null; tool_calls?: ToolCall[] };
  finish_reason: string | null;
}

Collecting Full Response

To collect the full streamed response into a single string:

const stream = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Tell me a story" }],
  stream: true,
});

let fullText = "";
for await (const chunk of stream) {
  fullText += chunk.choices[0]?.delta?.content ?? "";
}
console.log(fullText);

Aborting a Stream

Cancel an in-progress stream using the abort() method:

const stream = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Write a long essay" }],
  stream: true,
});

for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content ?? "";
  process.stdout.write(text);

  if (shouldStop) {
    stream.abort();
    break;
  }
}

Streaming with Tools

Tool calls are also streamed. The delta.tool_calls field accumulates across chunks:

const stream = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "What's the weather?" }],
  tools: [{ type: "function", function: { name: "get_weather", parameters: { type: "object", properties: { location: { type: "string" } } } } }],
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta;
  if (delta?.tool_calls) {
    // Accumulate tool call arguments across chunks
  }
  if (delta?.content) {
    process.stdout.write(delta.content);
  }
}