> ## Documentation Index
> Fetch the complete documentation index at: https://docs.anannas.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming

> Server-Sent Events streaming implementation

## Overview

Anannas supports streaming responses via Server-Sent Events (SSE). Enable streaming by setting `stream: true` in your request. The Anannas API returns incremental text deltas as tokens are generated, enabling real-time chat interfaces.

## Enabling Streaming

Set the `stream` parameter to `true`:

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "messages": [
    {"role": "user", "content": "Explain quantum computing"}
  ],
  "stream": true,
  "max_tokens": 500
}
```

## Response Format

Streaming responses use Server-Sent Events (SSE) with `Content-Type: text/event-stream`. Each event contains a JSON object following the OpenAI streaming format.

### Event Structure

Each SSE event has this format:

```
data: <json_object>

```

The JSON object structure:

```typescript theme={null}
type StreamChunk = {
  id: string;
  object: "chat.completion.chunk";
  created: number;
  model: string;
  choices: Array<{
    index: number;
    delta: {
      role?: "assistant";
      content?: string;
      tool_calls?: ToolCall[];
    };
    finish_reason?: "stop" | "length" | "tool_calls" | "content_filter" | null;
  }>;
};
```

### Example Stream

```
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{"content":"Quantum"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{"content":" computing"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]
```

## Implementation Examples

### Python

```python theme={null}
import requests
import json

response = requests.post(
  "https://api.anannas.ai/v1/chat/completions",
  headers={
    "Authorization": "Bearer <ANANNAS_API_KEY>",
    "Content-Type": "application/json",
  },
  json={
    "model": "openai/gpt-5-mini",
    "messages": [{"role": "user", "content": "Count to 5"}],
    "stream": True
  },
  stream=True
)

for line in response.iter_lines():
  if line:
    line = line.decode('utf-8')
    if line.startswith('data: '):
      data = line[6:]  # Remove 'data: ' prefix
      if data == '[DONE]':
        break
      try:
        chunk = json.loads(data)
        if chunk['choices'][0]['delta'].get('content'):
          print(chunk['choices'][0]['delta']['content'], end='', flush=True)
      except json.JSONDecodeError:
        pass
```

### TypeScript/JavaScript

```typescript theme={null}
const response = await fetch("https://api.anannas.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer <ANANNAS_API_KEY>",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "openai/gpt-5-mini",
    messages: [{ role: "user", content: "Count to 5" }],
    stream: true,
  }),
});

const reader = response.body?.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader!.read();
  if (done) break;

  const chunk = decoder.decode(value);
  const lines = chunk.split('\n');

  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const data = line.slice(6);
      if (data === '[DONE]') break;

      try {
        const json = JSON.parse(data);
        const content = json.choices[0]?.delta?.content;
        if (content) {
          process.stdout.write(content);
        }
      } catch (e) {
        // Ignore parse errors
      }
    }
  }
}
```

### Using OpenAI SDK

The OpenAI SDK handles streaming automatically:

```python theme={null}
from openai import OpenAI

client = OpenAI(
  base_url="https://api.anannas.ai/v1",
  api_key="<ANANNAS_API_KEY>"
)

stream = client.chat.completions.create(
  model="openai/gpt-5-mini",
  messages=[{"role": "user", "content": "Count to 5"}],
  stream=True
)

for chunk in stream:
  if chunk.choices[0].delta.content:
    print(chunk.choices[0].delta.content, end='', flush=True)
```

## SSE Comments

The stream may include comment lines for keep-alive:

```
: ANANNAS
```

These can be ignored per the [SSE specification](https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation). They serve as connection keep-alive signals.

## Cancellation

Cancel streaming requests by closing the connection. Billing stops immediately for supported providers when the connection is closed.

### Python

```python theme={null}
# Close the response stream
response.close()
```

### TypeScript

```typescript theme={null}
const controller = new AbortController();

fetch(url, {
  signal: controller.signal,
  // ... other options
});

// Cancel
controller.abort();
```

## Tool Calls in Streams

Tool calls are streamed incrementally. The `delta.tool_calls` array contains partial tool call information:

```json theme={null}
{
  "choices": [{
    "delta": {
      "tool_calls": [{
        "index": 0,
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\":\""
        }
      }]
    }
  }]
}
```

Accumulate `arguments` strings to reconstruct the complete JSON.

## Error Handling

Errors in streaming responses are sent as regular SSE events:

```
data: {"error": {"message": "Rate limit exceeded", "type": "rate_limit_error"}}
```

Handle errors by checking for the `error` field in parsed JSON objects.

## Finish Reasons

The final chunk includes `finish_reason`:

* `stop`: Natural completion or stop sequence
* `length`: Reached `max_tokens`
* `tool_calls`: Model requested tool execution
* `content_filter`: Content filtered by safety systems
* `null`: Stream incomplete

## Usage Tracking

Streaming responses include usage information in the final chunk or a separate event. Accumulate token counts across chunks for accurate usage tracking.

## Best Practices

1. **Buffer Management**: Accumulate deltas client-side for display
2. **Error Recovery**: Implement retry logic for network failures
3. **Connection Management**: Handle connection drops gracefully
4. **Token Counting**: Track usage from final chunk or usage event
5. **Cancellation**: Always cancel streams when user navigates away

## See Also

* [API Overview](/API/Overview) - Complete request/response schemas
* [Parameters](/API/Parameters) - All available parameters