# Anthropic API Compatibility
Source: https://docs.anannas.ai/API/Anthropic

Use Anannas with Anthropic-compatible tools and SDKs

## Overview

Anannas provides full compatibility with the Anthropic Messages API format, allowing you to use Anannas with any tool or SDK designed for Anthropic's API. This includes Claude Code, the official Anthropic SDK, and other tools in the Anthropic ecosystem.

The `/v1/messages` endpoint converts Anthropic Messages API requests to Anannas' internal format and works with **all models** that Anannas supports, not just Anthropic models. You can use any model (OpenAI, Google, Anthropic, etc.) through this endpoint by specifying the model in Anannas' `provider/model-name` format.

## Base URL

The Anannas Anthropic-compatible endpoint is available at:

```
https://api.anannas.ai/v1/messages
```

Alternatively, you can use:

```
https://api.anannas.ai/api/v1/messages
```

## Authentication

All requests require a Bearer token in the `Authorization` header using your Anannas API key:

```bash theme={null}
Authorization: Bearer <ANANNAS_API_KEY>
```

Get your API key from the [Anannas dashboard](https://anannas.ai/dashboard).

## Model IDs

The `/v1/messages` endpoint accepts **any model** that Anannas supports, not just Anthropic models. Specify models using Anannas' standard `provider/model-name` format. The API will automatically route to the appropriate provider.

### Example Models from Different Providers

**OpenAI Models:**

* `gpt-4.1-mini`
* `gpt-5-mini`

**Google Models:**

* `gemini-2.5-pro`
* `gemini-2.5-flash`

**Other Providers:**

* `grok-4-1-fast-non-reasoning-latest`
* `mistralai/mistral-large`
* `meta-llama/llama-3-70b`
* And many more...

For a complete list of available models, visit [anannas.ai/models](https://anannas.ai/models) or query `GET https://api.anannas.ai/v1/models`.

<Note>
  The endpoint converts Anthropic Messages API format to Anannas' internal format, allowing you to use any supported model through the Anthropic-compatible interface. Model routing and provider selection are handled automatically.
</Note>

## Using Claude Code

Claude Code is an agentic coding tool that lives in your terminal and helps you code faster through natural language commands. You can configure Claude Code to use Anannas instead of the default Anthropic API.

### Step 1: Install Claude Code

```bash theme={null}
npm install -g @anthropic-ai/claude-code
```

<Note>
  If you encounter permission issues during installation, try using `sudo` (macOS/Linux) or running the command prompt as an administrator (Windows).
</Note>

### Step 2: Configure Environment Variables

Set up environment variables to point Claude Code to Anannas:

**macOS/Linux:**

```bash theme={null}
export ANTHROPIC_BASE_URL=https://api.anannas.ai
export ANTHROPIC_AUTH_TOKEN=<ANANNAS_API_KEY>
export API_TIMEOUT_MS=600000
export ANTHROPIC_MODEL=grok-4-1-fast-non-reasoning-latest
export ANTHROPIC_SMALL_FAST_MODEL=gpt-5-mini
export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
```

**Windows (PowerShell):**

```powershell theme={null}
$env:ANTHROPIC_BASE_URL="https://api.anannas.ai"
$env:ANTHROPIC_AUTH_TOKEN="<ANANNAS_API_KEY>"
$env:API_TIMEOUT_MS="600000"
$env:ANTHROPIC_MODEL="grok-4-1-fast-non-reasoning-latest"
$env:ANTHROPIC_SMALL_FAST_MODEL="gpt-5-mini"
$env:CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC="1"
```

**Windows (Command Prompt):**

```cmd theme={null}
set ANTHROPIC_BASE_URL=https://api.anannas.ai
set ANTHROPIC_AUTH_TOKEN=<ANANNAS_API_KEY>
set API_TIMEOUT_MS=600000
set ANTHROPIC_MODEL=grok-4-1-fast-non-reasoning-latest
set ANTHROPIC_SMALL_FAST_MODEL=gpt-5-mini
set CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
```

<Note>
  The `API_TIMEOUT_MS` parameter is configured to prevent excessively long outputs that could cause the Claude Code client to time out. Here, we set the timeout duration to 10 minutes (600000ms).
</Note>

### Step 3: Start Claude Code

Navigate to your project directory and run:

```bash theme={null}
cd your-project
claude
```

If prompted with "Do you want to use this API key," select "Yes."

After launching, grant Claude Code permission to access files in your folder. You can now use Claude Code with Anannas.

### Model Configuration

Claude Code uses internal model environment variables that map to specific models. The default configuration maps:

* `ANTHROPIC_DEFAULT_OPUS_MODEL`: Maps to Claude Code's "opus" tier (most capable)
* `ANTHROPIC_DEFAULT_SONNET_MODEL`: Maps to Claude Code's "sonnet" tier (balanced)
* `ANTHROPIC_DEFAULT_HAIKU_MODEL`: Maps to Claude Code's "haiku" tier (fastest)

You can customize these mappings to use any Anannas-supported model. For example, you could use OpenAI models:

```bash theme={null}
export ANTHROPIC_DEFAULT_OPUS_MODEL=gpt-4.1-mini
export ANTHROPIC_DEFAULT_SONNET_MODEL=grok-4-1-fast-non-reasoning-latest
export ANTHROPIC_DEFAULT_HAIKU_MODEL=gpt-5-mini
```

Or mix providers:

```bash theme={null}
export ANTHROPIC_DEFAULT_OPUS_MODEL=anthropic/claude-opus-4
export ANTHROPIC_DEFAULT_SONNET_MODEL=grok-4-1-fast-non-reasoning-latest
export ANTHROPIC_DEFAULT_HAIKU_MODEL=gemini-2.5-flash
```

Alternatively, you can configure model mappings in Claude Code's settings file (`~/.claude/settings.json`):

```json theme={null}
{
  "env": {
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "gpt-5-mini",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "grok-4-1-fast-non-reasoning-latest",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "gpt-4.1-mini"
  }
}
```

<Note>
  It is generally not recommended to manually adjust the model mapping, as hardcoding the model mapping makes it inconvenient to automatically update to the latest model when Anannas adds new models. If you want to use the latest default mappings, simply delete the model mapping configuration in settings.json.
</Note>

## Using the Anthropic SDK

You can use the official Anthropic SDK with Anannas by configuring the base URL.

### Step 1: Install the Anthropic SDK

**Python:**

```bash theme={null}
pip install anthropic
```

**Node.js:**

```bash theme={null}
npm install @anthropic-ai/sdk
```

### Step 2: Configure the Client

**Python:**

```python theme={null}
import anthropic

client = anthropic.Anthropic(
    base_url="https://api.anannas.ai/v1",
    api_key="<ANANNAS_API_KEY>"
)
```

**Node.js:**

```typescript theme={null}
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  baseURL: 'https://api.anannas.ai/v1',
  apiKey: '<ANANNAS_API_KEY>',
});
```

### Step 3: Make API Calls

**Python:**

```python theme={null}
message = client.messages.create(
    model="grok-4-1-fast-non-reasoning-latest",
    max_tokens=1000,
    system="You are a helpful assistant.",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Hi, how are you?"
                }
            ]
        }
    ]
)

print(message.content)
```

**Node.js:**

```typescript theme={null}
const message = await client.messages.create({
  model: 'grok-4-1-fast-non-reasoning-latest',
  max_tokens: 1000,
  system: 'You are a helpful assistant.',
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'Hi, how are you?',
        },
      ],
    },
  ],
});

console.log(message.content);
```

## Direct HTTP Requests

You can also make direct HTTP requests to the endpoint:

<CodeGroup>
  ```python python theme={null}
  import requests

  response = requests.post(
      "https://api.anannas.ai/v1/messages",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
          "anthropic-version": "2023-06-01"
      },
      json={
          "model": "grok-4-1-fast-non-reasoning-latest",
          "max_tokens": 1000,
          "system": "You are a helpful assistant.",
          "messages": [
              {
                  "role": "user",
                  "content": [
                      {
                          "type": "text",
                          "text": "Explain quantum computing in simple terms"
                      }
                  ]
              }
          ]
      }
  )

  data = response.json()
  print(data["content"][0]["text"])
  ```

  ```typescript typescript theme={null}
  const response = await fetch('https://api.anannas.ai/v1/messages', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
      'anthropic-version': '2023-06-01',
    },
    body: JSON.stringify({
      model: 'grok-4-1-fast-non-reasoning-latest',
      max_tokens: 1000,
      system: 'You are a helpful assistant.',
      messages: [
        {
          role: 'user',
          content: [
            {
              type: 'text',
              text: 'Explain quantum computing in simple terms',
            },
          ],
        },
      ],
    }),
  });

  const data = await response.json();
  console.log(data.content[0].text);
  ```

  ```bash bash theme={null}
  curl https://api.anannas.ai/v1/messages \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $ANANNAS_API_KEY" \
    -H "anthropic-version: 2023-06-01" \
    -d '{
      "model": "grok-4-1-fast-non-reasoning-latest",
      "max_tokens": 1000,
      "system": "You are a helpful assistant.",
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "Explain quantum computing in simple terms"
            }
          ]
        }
      ]
    }'
  ```
</CodeGroup>

## API Compatibility Details

### HTTP Headers

| Field               | Support Status                                        |
| ------------------- | ----------------------------------------------------- |
| `anthropic-beta`    | Ignored                                               |
| `anthropic-version` | Ignored                                               |
| `x-api-key`         | Fully Supported (use `Authorization: Bearer` instead) |
| `Authorization`     | Fully Supported                                       |

### Request Fields

| Field            | Support Status      | Notes                                                                                                                                                      |
| ---------------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model`          | Fully Supported     | Use any Anannas model format (e.g., `grok-4-1-fast-non-reasoning-latest`, `anthropic/claude-4.5-sonnet`, `gemini-2.5-pro`)                                 |
| `max_tokens`     | Fully Supported     | Required field                                                                                                                                             |
| `messages`       | Fully Supported     | Required field, minimum 1 message                                                                                                                          |
| `system`         | Supported           | String or array of system blocks (cache\_control in array blocks is not preserved)                                                                         |
| `temperature`    | Fully Supported     | Range \[0.0 \~ 2.0]                                                                                                                                        |
| `top_p`          | Fully Supported     | Range \[0.0 \~ 1.0]                                                                                                                                        |
| `top_k`          | Ignored             | Not supported by all models. Not allowed when `thinking` is enabled                                                                                        |
| `stop_sequences` | Fully Supported     | Array of stop sequences                                                                                                                                    |
| `stream`         | Fully Supported     | Server-Sent Events (SSE)                                                                                                                                   |
| `tools`          | Fully Supported     | Tool definitions for function calling                                                                                                                      |
| `tool_choice`    | Fully Supported     | `none`, `auto`, `any`, or specific tool. When `thinking` is enabled, only `"auto"` or `"none"` are allowed                                                 |
| `thinking`       | Supported           | `budget_tokens` is converted to `reasoning.max_tokens`. Interleaved thinking is automatically enabled when tools are present and `tool_choice` is `"auto"` |
| `metadata`       | Partially Supported | `user_id` is supported                                                                                                                                     |

### Tool Fields

| Field           | Support Status  |                                  |
| --------------- | --------------- | -------------------------------- |
| `name`          | Fully Supported |                                  |
| `description`   | Fully Supported |                                  |
| `input_schema`  | Fully Supported |                                  |
| `cache_control` | Supported       | Prompt caching support for tools |

### Tool Choice Values

| Value  | Support Status  | Notes                                                                                                |
| ------ | --------------- | ---------------------------------------------------------------------------------------------------- |
| `none` | Fully Supported |                                                                                                      |
| `auto` | Supported       | `disable_parallel_tool_use` is ignored. Required for interleaved thinking when `thinking` is enabled |
| `any`  | Supported       | `disable_parallel_tool_use` is ignored. Not allowed when `thinking` is enabled                       |
| `tool` | Supported       | `disable_parallel_tool_use` is ignored. Not allowed when `thinking` is enabled                       |

### Message Content Types

| Content Type        | Support Status  | Notes                                                          |
| ------------------- | --------------- | -------------------------------------------------------------- |
| `text`              | Fully Supported | String or content block                                        |
| `image`             | Supported       | Base64-encoded images                                          |
| `tool_use`          | Fully Supported | Tool call from assistant                                       |
| `tool_result`       | Fully Supported | Tool result from user                                          |
| `thinking`          | Supported       | Reasoning/thinking blocks                                      |
| `document`          | Not Supported   |                                                                |
| `search_result`     | Not Supported   |                                                                |
| `redacted_thinking` | Supported       | Redacted/encrypted thinking blocks are supported and preserved |
| `server_tool_use`   | Not Supported   |                                                                |
| `mcp_tool_use`      | Not Supported   |                                                                |

### Response Format

Responses follow the Anthropic Messages API format:

```json theme={null}
{
  "id": "msg_abc123",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Response text here"
    }
  ],
  "model": "grok-4-1-fast-non-reasoning-latest",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 10,
    "output_tokens": 50
  }
}
```

### Streaming

When `stream: true` is set, the API returns Server-Sent Events (SSE) with the following event types:

* `message_start`: Initial message metadata
* `content_block_start`: Start of a content block
* `content_block_delta`: Incremental content updates
* `content_block_stop`: End of a content block
* `message_delta`: Message-level updates (stop\_reason, usage)
* `message_stop`: End of the message stream
* `ping`: Keep-alive heartbeat events
* `error`: Error events

## Troubleshooting

### Claude Code Configuration Not Working

If you manually modified the `~/.claude/settings.json` configuration file but found the changes did not take effect:

1. Close all Claude Code windows, open a new command-line window, and run `claude` again
2. If the issue persists, try deleting the `~/.claude/settings.json` file and then reconfigure the environment variables
3. Confirm that the JSON format of the configuration file is correct - check variable names and ensure there are no missing or extra commas

### Model Not Found Errors

If you receive a "model not found" error:

1. Verify the model ID format is correct: `provider/model-name` (e.g., `grok-4-1-fast-non-reasoning-latest`, `anthropic/claude-3.7-sonnet`)
2. Check available models at [anannas.ai/models](https://anannas.ai/models) or via `GET https://api.anannas.ai/v1/models`
3. Ensure your API key has access to the requested model
4. Remember: you can use any model supported by Anannas, not just Anthropic models

### Authentication Errors

If you receive authentication errors:

1. Verify your API key is correct and active in the [Anannas dashboard](https://anannas.ai/dashboard)
2. Ensure you're using `Authorization: Bearer <ANANNAS_API_KEY>` header format
3. Check that your API key has sufficient credits

## Next Steps

* Explore [Streaming](/API/Streaming) for real-time responses
* Learn about [Tool Calling](/Features/tool-calling) for function execution
* Review [Reasoning](/UseCases/reasoning) for thinking/reasoning capabilities
* Check [Authentication](/API/Authentication) for security best practices


# Authentication
Source: https://docs.anannas.ai/API/Authentication

API authentication and security

## Overview

Anannas uses Bearer token authentication via the `Authorization` header. All API requests require a valid API key.

## API Keys

API keys are generated in the [Anannas dashboard](https://anannas.ai/dashboard). Each key is associated with:

* User account and organization
* Credit limits and billing
* Rate limits based on tier
* Usage tracking and analytics

## Authentication Header

Include your API key in the `Authorization` header:

```bash theme={null}
Authorization: Bearer <ANANNAS_API_KEY>
```

### Example

```python theme={null}
import requests

response = requests.post(
  "https://api.anannas.ai/v1/chat/completions",
  headers={
    "Authorization": "Bearer sk-...",
    "Content-Type": "application/json",
  },
  json={
    "model": "openai/gpt-5-mini",
    "messages": [{"role": "user", "content": "Hello"}]
  }
)
```

## Security Best Practices

### Store Keys Securely

**Never commit API keys to version control.** Use environment variables:

```python theme={null}
import os
api_key = os.getenv("ANANNAS_API_KEY")
```

```typescript theme={null}
const apiKey = process.env.ANANNAS_API_KEY;
```

```bash theme={null}
export ANANNAS_API_KEY="sk-..."
curl -H "Authorization: Bearer $ANANNAS_API_KEY" ...
```

### Key Rotation

If a key is compromised:

1. Immediately revoke the key in the dashboard
2. Generate a new key
3. Update your application configuration
4. Monitor for unauthorized usage

### Key Scope

API keys have full access to:

* All models available to your account
* Credit balance and billing
* Usage analytics
* Rate limit quotas

Restrict key access at the application level if needed.

## Error Responses

### 401 Unauthorized

Invalid or missing API key:

```json theme={null}
{
  "error": {
    "message": "Authentication required",
    "type": "authentication_error"
  }
}
```

Common causes:

* Missing `Authorization` header
* Invalid key format
* Revoked or expired key
* Key not associated with active account

### 402 Payment Required

Insufficient credits:

```json theme={null}
{
  "error": {
    "message": "Insufficient credits",
    "type": "insufficient_quota_error"
  }
}
```

## Multiple Keys

You can create multiple API keys for:

* Different environments (development, staging, production)
* Different applications or services
* Team members with separate usage tracking

Each key maintains independent usage statistics.

## Key Management

Manage keys in the dashboard:

* Create new keys
* Revoke existing keys
* View key usage and statistics
* Set key-specific rate limits (if supported)

## Testing Authentication

Verify your API key:

```bash theme={null}
curl https://api.anannas.ai/v1/models \
  -H "Authorization: Bearer $ANANNAS_API_KEY"
```

A successful response confirms authentication is working.


# Rate Limits
Source: https://docs.anannas.ai/API/LIMITS

Rate limits, quotas, and usage tracking

## Overview

Rate limits are tier-based and applied per API key. Limits vary by account tier and model. Creating additional accounts or keys does not bypass global capacity limits.

## Checking Limits

Query your API key status:

```
GET https://api.anannas.ai/v1/key
```

### Query Parameters

* `details=true`: Include detailed usage statistics

### Basic Request

```python theme={null}
import requests

response = requests.get(
  "https://api.anannas.ai/v1/key",
  headers={"Authorization": "Bearer <ANANNAS_API_KEY>"}
)

data = response.json()
print(f"Usage: {data['data']['usage']}")
print(f"Limit: {data['data']['limit']}")
```

### Detailed Request

```python theme={null}
response = requests.get(
  "https://api.anannas.ai/v1/key?details=true",
  headers={"Authorization": "Bearer <ANANNAS_API_KEY>"}
)

data = response.json()
print(f"Balance: {data['data']['balance']}")
print(f"Usage stats: {data['data']['usage_stats']}")
```

## Response Format

### Basic Response

```json theme={null}
{
  "data": {
    "created_at": "2024-01-01T00:00:00Z",
    "is_active": true,
    "is_org_key": true,
    "label": "Main Key",
    "last_used": "2024-01-15T10:30:00Z",
    "limit": 1000,
    "name": "My API Key",
    "org_id": "691744a8236e7e5afac6564c",
    "org_name": "My Organization",
    "tier": "tier2",
    "tier_name": "Standard",
    "usage": 120.5
  }
}
```

### Detailed Response

With `details=true`:

```json theme={null}
{
  "data": {
    "account_type": "organization",
    "balance": 112.85,
    "created_at": "2024-01-01T00:00:00Z",
    "is_active": true,
    "is_org_key": true,
    "label": "Main Key",
    "last_used": "2024-01-15T10:30:00Z",
    "limit": 1000,
    "name": "My API Key",
    "org_id": "691744a8236e7e5afac6564c",
    "org_name": "My Organization",
    "tier": "tier2",
    "tier_name": "Standard",
    "usage": 120.5,
    "usage_stats": {
      "last_30_days": {
        "cost": 0.11,
        "requests": 16
      },
      "last_7_days": {
        "cost": 0.11,
        "requests": 16
      }
    }
  }
}
```

## Response Fields

| Field          | Type             | Description                                 |
| -------------- | ---------------- | ------------------------------------------- |
| `usage`        | `number`         | Total credits used (float)                  |
| `limit`        | `number \| null` | Credit limit (null if unlimited)            |
| `balance`      | `number`         | Current account balance (details only)      |
| `account_type` | `string`         | `"organization"` or `"user"` (details only) |
| `tier`         | `string`         | Tier identifier (e.g., `"tier2"`)           |
| `tier_name`    | `string`         | Human-readable tier name                    |
| `org_id`       | `string`         | Organization ID (if applicable)             |
| `org_name`     | `string`         | Organization name (if applicable)           |
| `is_org_key`   | `boolean`        | Whether this is an organization-level key   |
| `is_active`    | `boolean`        | Whether the key is active                   |
| `label`        | `string`         | Optional key label                          |
| `name`         | `string`         | Key name                                    |
| `created_at`   | `string`         | ISO 8601 timestamp                          |
| `last_used`    | `string`         | ISO 8601 timestamp                          |
| `usage_stats`  | `object`         | Usage statistics (details only)             |

### Usage Statistics

`usage_stats` contains:

* `last_7_days`: Object with `cost` and `requests`
* `last_30_days`: Object with `cost` and `requests`

## Rate Limit Behavior

### 429 Too Many Requests

When rate limits are exceeded:

```json theme={null}
{
  "error": {
    "message": "Rate limit exceeded",
    "type": "rate_limit_error"
  }
}
```

### Rate Limit Headers

Responses may include rate limit headers (if supported):

```
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 950
X-RateLimit-Reset: 1694268190
```

## Tier-Based Limits

Limits vary by account tier:

* **Free Tier**: Lower limits, daily caps
* **Standard (Tier 2)**: Higher limits
* **Enterprise**: Custom limits

Check your tier via the `/v1/key` endpoint.

## Model-Specific Limits

Different models may have different rate limits. Distribute usage across models if needed to maximize throughput.

## Free Tier Limits

Free-tier models have additional restrictions:

* Daily request caps
* Per-minute request limits
* Model availability restrictions

## Negative Balance

If account balance is below zero, all requests (including free-tier) fail with `402 Payment Required` until credits are added.

## DDoS Protection

Excessive request bursts may be blocked. Implement:

* Request throttling
* Exponential backoff
* Rate limit monitoring

## Monitoring Usage

### Programmatic Monitoring

```python theme={null}
import requests
import time

def check_usage(api_key):
  response = requests.get(
    "https://api.anannas.ai/v1/key?details=true",
    headers={"Authorization": f"Bearer {api_key}"}
  )
  data = response.json()["data"]
  
  usage_pct = (data["usage"] / data["limit"]) * 100 if data["limit"] else 0
  print(f"Usage: {data['usage']}/{data['limit']} ({usage_pct:.1f}%)")
  print(f"Balance: ${data['balance']:.2f}")
  
  return data

# Monitor every 5 minutes
while True:
  check_usage(api_key)
  time.sleep(300)
```

## Best Practices

1. **Monitor Regularly**: Check usage via `/v1/key` endpoint
2. **Handle 429**: Implement exponential backoff
3. **Distribute Load**: Use multiple models to maximize throughput
4. **Set Alerts**: Monitor balance and usage thresholds
5. **Respect Limits**: Don't attempt to bypass rate limits

## See Also

* [Authentication](/API/Authentication) - API key management
* [Errors](/API/errors) - Error handling


# API Reference
Source: https://docs.anannas.ai/API/Overview

Complete API reference for Anannas Chat Completions

## Endpoint

```
POST https://api.anannas.ai/v1/chat/completions
```

## Request Schema

The Anannas API follows the OpenAI Chat Completions format with Anannas-specific extensions. The request body is JSON.

### Required Fields

* `model` (string): Model identifier in format `provider/model-name` (e.g., `openai/gpt-5-mini`, `anthropic/claude-3-sonnet`)
* `messages` (array): Array of message objects. Minimum 1 message required.

### Request Type Definition

```typescript theme={null}
type ChatCompletionRequest = {
  // Required
  model: string;
  messages: Message[];

  // Optional: Sampling parameters
  temperature?: number;        // 0.0-2.0, default: 1.0
  top_p?: number;              // 0.0-1.0, default: 1.0
  max_tokens?: number;         // Maximum tokens to generate
  max_completion_tokens?: number; // Alternative to max_tokens
  stop?: string | string[];    // Stop sequences
  seed?: number;               // For deterministic outputs
  frequency_penalty?: number;  // -2.0 to 2.0
  presence_penalty?: number;   // -2.0 to 2.0
  top_k?: number;              // 0 or above
  repetition_penalty?: number; // (0, 2]
  min_p?: number;              // [0, 1]
  top_a?: number;              // [0, 1]
  logit_bias?: { [token_id: number]: number }; // Token bias map

  // Optional: Streaming
  stream?: boolean;            // Enable Server-Sent Events

  // Optional: Tool calling (OpenAI-compatible)
  tools?: Tool[];
  tool_choice?: "none" | "auto" | "required" | { type: "function", function: { name: string } };
  parallel_tool_calls?: boolean; // Default: true

  // Optional: Structured outputs
  response_format?: {
    type: "json_object";
  } | {
    type: "json_schema";
    json_schema: {
      name: string;
      strict?: boolean;
      schema: object; // JSON Schema object
    };
  };

  // Optional: Multimodal
  modalities?: string[];       // ["text", "audio", "image"]
  audio?: AudioConfig;         // Audio output configuration

  // Optional: Reasoning (for o1, o3, Claude Sonnet 4.5, etc.)
  reasoning?: {
    effort?: "low" | "medium" | "high";
    max_tokens?: number;
    enabled?: boolean;
    exclude?: boolean;
  };
  thinking_config?: {          // External API alias for reasoning
    include_thoughts?: boolean;
    thinking_budget?: number;
    thinking_level?: string;
  };

  // Optional: Anannas-specific routing
  models?: string[];           // Model routing fallbacks
  route?: "fallback";          // Enable smart fallback routing
  provider?: {
    order?: string[];           // Provider preference order
    allow_fallbacks?: boolean;  // Default: true
    require_parameters?: boolean;
    data_collection?: "allow" | "deny";
    zdr?: boolean;              // Zero Data Retention only
    only?: string[];            // Whitelist providers
    ignore?: string[];          // Blacklist providers
    quantizations?: string[];   // Filter by quantization
    sort?: "price" | "throughput" | "latency";
    max_price?: {
      prompt?: number;
      completion?: number;
      request?: number;
      image?: number;
    };
  };
  fallbacks?: Array<string | {
    model: string;
    provider?: ProviderPreferences;
    metadata?: { [key: string]: string };
    reason?: string;
  }>;

  // Optional: User tracking
  user?: string;               // Stable user identifier

  // Optional: Prompt caching
  prompt_cache_key?: string;   // OpenAI prompt caching

  // Optional: Metadata
  metadata?: { [key: string]: string }; // Custom tracking data

  // Optional: Plugins (PDF support)
  plugins?: Array<{
    type: string;
    [key: string]: any;
  }>;

  // Optional: Grok-specific
  search_parameters?: {
    mode?: "off" | "on" | "auto";
    max_search_results?: number; // 1-30
    from_date?: string;          // ISO-8601
    to_date?: string;            // ISO-8601
    return_citations?: boolean;
    sources?: Array<{
      type: "x" | "web" | "news" | "rss" | "live_search";
      included_x_handles?: string[];
      excluded_x_handles?: string[];
      allowed_websites?: string[];
      excluded_websites?: string[];
      country?: string;
      safe_search?: boolean;
      links?: string[];
    }>;
  };

  // Optional: MCP (Model Context Protocol)
  mcp?: {
    servers: Array<{
      name: string;
      [key: string]: any;
    }>;
  };

  // Deprecated: Use messages instead
  prompt?: string;
};
```

### Message Object

```typescript theme={null}
type Message = {
  role: "system" | "user" | "assistant" | "tool";
  content: string | ContentPart[];
  name?: string;               // For named tools/functions
  tool_call_id?: string;      // For tool result messages
  tool_calls?: ToolCall[];     // Assistant tool invocations
};

type ContentPart = {
  type: "text" | "image_url" | "file" | "input_audio";
  text?: string;
  image_url?: {
    url: string;               // URL or base64 data URI
    detail?: "low" | "high" | "auto";
  };
  file?: {
    url?: string;
    data?: string;             // Base64
    type?: string;             // "application/pdf"
  };
  input_audio?: {
    data: string;              // Base64 audio
    format: string;            // "wav", "mp3", etc.
  };
  cache_control?: {            // Anthropic caching
    type: "ephemeral";
    ttl: string;               // "5m"
  };
};
```

### Example Request

```python theme={null}
import requests

response = requests.post(
  "https://api.anannas.ai/v1/chat/completions",
  headers={
    "Authorization": "Bearer <ANANNAS_API_KEY>",
    "Content-Type": "application/json",
  },
  json={
    "model": "openai/gpt-5-mini",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Explain quantum computing"}
    ],
    "temperature": 0.7,
    "max_tokens": 500,
    "stream": False
  }
)
```

## Response Schema

Responses follow the OpenAI Chat Completions format:

```typescript theme={null}
type ChatCompletionResponse = {
  id: string;                  // Unique completion ID
  object: "chat.completion" | "chat.completion.chunk";
  created: number;              // Unix timestamp
  model: string;                // Model identifier used
  choices: Array<{
    index: number;
    message: {
      role: "assistant";
      content: string | null;
      tool_calls?: ToolCall[];
      refusal?: string | null;  // Content refusal (Anthropic)
      reasoning?: string;       // Reasoning tokens (o1, o3, etc.)
      audio?: AudioOutput;
      images?: ImageOutput[];
      citations?: string[];    // Grok citations
    };
    finish_reason: "stop" | "length" | "tool_calls" | "content_filter" | "null";
    delta?: {                  // Streaming only
      role?: "assistant";
      content?: string;
      tool_calls?: ToolCall[];
    };
  }>;
  usage: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
    cache_read_input_tokens?: number;      // Cached tokens read
    cache_creation_input_tokens?: number;  // Cache creation cost (Anthropic)
    reasoning_tokens?: number;            // Reasoning token count
    audio_tokens?: number;                 // Audio token count
  };
  system_fingerprint?: string;
};
```

### Example Response

```json theme={null}
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "openai/gpt-5-mini",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Quantum computing is a computational paradigm..."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 150,
    "total_tokens": 165
  }
}
```

## Headers

### Required

* `Authorization: Bearer <ANANNAS_API_KEY>` - API key authentication
* `Content-Type: application/json` - Request content type

### Optional

* `HTTP-Referer: <YOUR_SITE_URL>` - Identifies your application
* `X-Title: <YOUR_APP_NAME>` - Sets application name for analytics

## Finish Reasons

The `finish_reason` field indicates why generation stopped:

* `stop`: Model generated a stop sequence or natural completion
* `length`: Reached `max_tokens` limit
* `tool_calls`: Model requested tool execution
* `content_filter`: Content was filtered by safety systems
* `null`: Generation incomplete (streaming)

## Prompt Caching

<Card title="Check Caching Support" icon="book">
  For models that support prompt caching and current pricing, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

### OpenAI Models

Use `prompt_cache_key` to cache prompt prefixes:

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is 2+2?"}
  ],
  "prompt_cache_key": "system-prompt-v1"
}
```

**Pricing:**

* Cache reads: 50% of input token price
* Cache writes: No additional cost

### Anthropic Models

Use `cache_control` in message content parts:

```json theme={null}
{
  "model": "anthropic/claude-3-sonnet",
  "messages": [{
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "You are a helpful assistant.",
        "cache_control": {
          "type": "ephemeral",
          "ttl": "5m"
        }
      },
      {
        "type": "text",
        "text": "What is 2+2?"
      }
    ]
  }]
}
```

**Pricing:**

* Cache creation: 1.25x input token price
* Cache reads: 0.1x input token price (90% discount)

<Card title="Verify Caching Pricing" icon="book">
  For current caching pricing and supported models, check [anannas.ai/models](https://anannas.ai/models).
</Card>

**Limits:**

* Maximum 4 content blocks with `cache_control` per request
* Cache expires after 5 minutes

## Error Responses

Errors follow this format:

```json theme={null}
{
  "error": {
    "message": "Error description",
    "type": "error_type",
    "code": "error_code"
  }
}
```

Common error types:

* `invalid_request_error`: Malformed request, missing required fields
* `authentication_error`: Invalid or missing API key
* `rate_limit_error`: Rate limit exceeded
* `insufficient_quota_error`: Insufficient credits (402)
* `server_error`: Internal server error

## Model Routing

If `model` is omitted, Anannas selects the default model for your account. The routing system automatically:

1. Selects optimal provider based on price, availability, and latency
2. Falls back to alternative providers if primary fails
3. Respects `provider` preferences when specified

Use `fallbacks` for explicit cross-model fallback chains:

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "fallbacks": [
    "anthropic/claude-3-sonnet",
    "openai/gpt-3.5-turbo"
  ],
  "messages": [...]
}
```

## See Also

* [Parameters](/API/Parameters) - Complete parameter reference
* [Streaming](/API/Streaming) - Server-Sent Events implementation
* [Authentication](/API/Authentication) - Security and API keys
* [Models](/models) - Available models and capabilities


# Parameters
Source: https://docs.anannas.ai/API/Parameters

Complete parameter reference for chat completions

## Overview

This document describes all parameters available in the `/v1/chat/completions` endpoint. Parameters are validated server-side; unsupported parameters for a given provider are ignored.

## Required Parameters

### `model`

**Type:** `string`\
**Required:** Yes\
**Description:** Model identifier in format `provider/model-name`

Examples:

* `openai/gpt-5-mini`
* `anthropic/claude-3-sonnet`
* `openai/gpt-3.5-turbo`

### `messages`

**Type:** `Message[]`\
**Required:** Yes\
**Minimum:** 1 message\
**Description:** Array of message objects with `role` and `content`

Message roles:

* `system`: System instructions (typically first message)
* `user`: User input
* `assistant`: Model responses (for conversation history)
* `tool`: Tool execution results

## Sampling Parameters

### `temperature`

**Type:** `number`\
**Range:** `0.0` - `2.0`\
**Default:** `1.0`\
**Description:** Controls randomness. Lower values make output more deterministic.

* `0.0`: Most deterministic
* `1.0`: Balanced
* `2.0`: Most random

### `top_p`

**Type:** `number`\
**Range:** `0.0` - `1.0`\
**Default:** `1.0`\
**Description:** Nucleus sampling - considers tokens with cumulative probability mass.

### `max_tokens`

**Type:** `integer`\
**Minimum:** `1`\
**Description:** Maximum tokens to generate. Model-specific limits apply.

### `max_completion_tokens`

**Type:** `integer`\
**Description:** Alternative to `max_tokens` (provider-specific).

### `stop`

**Type:** `string | string[]`\
**Description:** Stop sequences that halt generation. Can be a single string or array.

```json theme={null}
{
  "stop": ["\n\n", "Human:"]
}
```

### `seed`

**Type:** `integer`\
**Description:** Random seed for deterministic outputs. Only supported by some models.

## Penalties

### `frequency_penalty`

**Type:** `number`\
**Range:** `-2.0` - `2.0`\
**Description:** Reduces likelihood of repeating tokens. Positive values decrease repetition.

### `presence_penalty`

**Type:** `number`\
**Range:** `-2.0` - `2.0`\
**Description:** Reduces likelihood of discussing new topics. Positive values encourage new topics.

### `repetition_penalty`

**Type:** `number`\
**Range:** `(0, 2]`\
**Description:** Provider-specific repetition control.

## Advanced Sampling

### `top_k`

**Type:** `integer`\
**Minimum:** `0`\
**Description:** Limits sampling to top K tokens by probability.

<Note>
  **Not allowed with reasoning**: When `reasoning` is enabled, `top_k` is not supported and will result in an error.
</Note>

### `top_a`

**Type:** `number`\
**Range:** `[0, 1]`\
**Description:** Provider-specific sampling parameter.

### `min_p`

**Type:** `number`\
**Range:** `[0, 1]`\
**Description:** Minimum probability threshold for token selection.

### `logit_bias`

**Type:** `{ [token_id: number]: number }`\
**Description:** Bias specific tokens by ID. Values typically range from -100 to 100.

```json theme={null}
{
  "logit_bias": {
    "1234": 10,   // Increase likelihood
    "5678": -10   // Decrease likelihood
  }
}
```

## Tool Calling

### `tools`

**Type:** `Tool[]`\
**Description:** Array of function definitions for tool calling.

```json theme={null}
{
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get weather for a location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {"type": "string"}
        },
        "required": ["location"]
      }
    }
  }]
}
```

### `tool_choice`

**Type:** `"none" | "auto" | "required" | { type: "function", function: { name: string } }`\
**Default:** `"auto"`\
**Description:** Controls tool usage behavior.

* `"none"`: No tools called
* `"auto"`: Model decides
* `"required"`: Model must call a tool
* Object: Force specific function

<Note>
  **Restrictions with reasoning**: When `reasoning` is enabled, only `"auto"` or `"none"` are allowed. Using `"required"` or forcing a specific tool will result in an error. This restriction enables interleaved thinking, which allows reasoning between tool calls.
</Note>

### `parallel_tool_calls`

**Type:** `boolean`\
**Default:** `true`\
**Description:** Allow multiple tool calls in a single response.

## Structured Outputs

### `response_format`

**Type:** `{ type: "json_object" } | { type: "json_schema", json_schema: object }`\
**Description:** Enforce JSON output format.

**JSON Object:**

```json theme={null}
{
  "response_format": {
    "type": "json_object"
  }
}
```

**JSON Schema:**

```json theme={null}
{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "response",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "answer": {"type": "string"}
        }
      }
    }
  }
}
```

## Streaming

### `stream`

**Type:** `boolean`\
**Default:** `false`\
**Description:** Enable Server-Sent Events streaming. See [Streaming documentation](/API/Streaming).

## Reasoning

### `reasoning`

**Type:** `object`\
**Description:** Configure reasoning for models that support it.

<Card title="Check Reasoning Support" icon="book">
  For models that support reasoning and their configuration options, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

```json theme={null}
{
  "reasoning": {
    "effort": "high",      // "low" | "medium" | "high"
    "max_tokens": 10000,   // Maximum reasoning tokens
    "enabled": true,       // Enable reasoning
    "exclude": false       // Exclude from response
  }
}
```

<Note>
  **Interleaved Thinking with Tools**

  When using reasoning with tools on Claude 4 models (Sonnet 4.5, Opus 4.5, Haiku 4.5), interleaved thinking is automatically enabled. This allows the model to reason between tool calls. With interleaved thinking, `max_tokens` in the reasoning config can exceed the request's `max_tokens` parameter, as it represents the total budget across all thinking blocks within one assistant turn.

  Interleaved thinking requires `tool_choice: "auto"` (or no tool\_choice specified). Using `tool_choice: "required"` or forcing a specific tool will result in an error when reasoning is enabled.
</Note>

### `thinking_config`

**Type:** `object`\
**Description:** External API alias for `reasoning`. Maps to `reasoning` internally.

```json theme={null}
{
  "thinking_config": {
    "include_thoughts": true,
    "thinking_budget": 10000,
    "thinking_level": "high"
  }
}
```

## Multimodal

### `modalities`

**Type:** `string[]`\
**Description:** Requested output modalities: `["text", "audio", "image"]`

### `audio`

**Type:** `object`\
**Description:** Audio output configuration.

```json theme={null}
{
  "audio": {
    "voice": "alloy",
    "format": "mp3"
  }
}
```

## Prompt Caching

### `prompt_cache_key`

**Type:** `string`\
**Description:** Cache key for OpenAI prompt caching. See [Overview](/API/Overview#prompt-caching).

## Routing

### `models`

**Type:** `string[]`\
**Description:** Fallback model list for routing.

### `route`

**Type:** `"fallback"`\
**Description:** Enable smart fallback routing.

### `provider`

**Type:** `object`\
**Description:** Provider selection preferences.

<Card title="Check Provider Pricing" icon="book">
  For current pricing to set `max_price` limits, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

```json theme={null}
{
  "provider": {
    "order": ["openai", "anthropic"],
    "allow_fallbacks": true,
    "require_parameters": false,
    "data_collection": "deny",
    "zdr": true,
    "only": ["openai"],
    "ignore": ["anthropic"],
    "quantizations": ["q4", "q8"],
    "sort": "price",
    "max_price": {
      "prompt": 0.001,
      "completion": 0.002
    }
  }
}
```

### `fallbacks`

**Type:** `Array<string | object>`\
**Description:** Explicit fallback chain.

```json theme={null}
{
  "fallbacks": [
    "anthropic/claude-3-sonnet",
    {
      "model": "openai/gpt-3.5-turbo",
      "provider": {"only": ["openai"]}
    }
  ]
}
```

## User Tracking

### `user`

**Type:** `string`\
**Description:** Stable identifier for end-users (abuse prevention).

## Metadata

### `metadata`

**Type:** `{ [key: string]: string }`\
**Description:** Custom metadata for request tracking.

```json theme={null}
{
  "metadata": {
    "request_id": "req-123",
    "environment": "production"
  }
}
```

## Provider-Specific Parameters

<Card title="Check Parameter Support" icon="book">
  For detailed parameter support by model and provider, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

Some parameters are provider-specific and may be ignored by others:

* **Mistral**: `safe_prompt`
* **Hyperbolic**: `raw_mode`
* **Grok**: `search_parameters`, `deferred`
* **Anthropic**: `cache_control` in message content

## Parameter Validation

* Invalid parameter values return `400 Bad Request`
* Unsupported parameters are silently ignored
* Provider-specific parameters are passed through when supported

## See Also

* [API Overview](/API/Overview) - Request/response schemas
* [Streaming](/API/Streaming) - Streaming implementation
* [Models](/models) - Model capabilities and parameter support


# Responses API
Source: https://docs.anannas.ai/API/Responses

An overview of Anannas's Responses API.

The Anannas Responses API provides a unified interface for building advanced AI agents capable of executing complex tasks autonomously. This API is compatible with OpenAI's Responses API format and supports multimodal inputs, reasoning capabilities, and seamless tool integration.

<Warning>
  **Stateless API Implementation**

  This API implements the **stateless version** of the Responses API. Unlike OpenAI's stateful Responses API, Anannas does not maintain conversation state between requests. Each request is completely independent, and you must include the full conversation history in every request.
</Warning>

<Note>
  **Beta API**

  This API is in **beta stage** and may have breaking changes. Use with caution in production environments.
</Note>

### Base URL

```
https://api.anannas.ai/api/v1/responses
```

### Authentication

All requests require authentication using your Anannas API key:

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": "Hello, world!",
      },
  )
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: 'Hello, world!',
    }),
  });
  ```

  ```go Go theme={null}
  package main

  import (
  	"bytes"
  	"encoding/json"
  	"fmt"
  	"net/http"
  )

  func main() {
  	reqBody := map[string]interface{}{
  		"model": "openai/gpt-5-mini",
  		"input": "Hello, world!",
  	}

  	jsonData, _ := json.Marshal(reqBody)
  	req, _ := http.NewRequest("POST", "https://api.anannas.ai/api/v1/responses", bytes.NewBuffer(jsonData))
  	req.Header.Set("Authorization", "Bearer <ANANNAS_API_KEY>")
  	req.Header.Set("Content-Type", "application/json")

  	client := &http.Client{}
  	resp, err := client.Do(req)
  	if err != nil {
  		fmt.Println("Error:", err)
  		return
  	}
  	defer resp.Body.Close()
  }
  ```

  ```bash cURL theme={null}
  curl https://api.anannas.ai/api/v1/responses \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer <ANANNAS_API_KEY>" \
    -d '{
      "model": "openai/gpt-5-mini",
      "input": "Hello, world!"
    }'
  ```
</CodeGroup>

### Core Features

* **Stateless Design**: Each request is independent - you manage conversation history client-side
* **Multimodal Support**: Handle various input types, including text, images, and audio
* **Reasoning Capabilities**: Access advanced reasoning with configurable effort levels
* **Tool Integration**: Utilize function calling with support for parallel execution
* **Streaming Support**: Receive responses in real-time as they're generated

<Info>
  **Managing Conversation History**

  Since this API is stateless, you must include the complete conversation history in each request. Include all previous user messages and assistant responses in the `input` array to maintain context.
</Info>

### Request Format

The Responses API uses a structured request format with an `input` array containing conversation messages:

<CodeGroup>
  ```typescript TypeScript theme={null}
  type ResponsesRequest = {
    // Required
    model: string;
    input: ResponsesMessage[];

    // Optional
    instructions?: string;
    response_format?: { type: 'json_object' };
    metadata?: { [key: string]: string };
    temperature?: number;
    max_output_tokens?: number;
    stream?: boolean;
    
    // Tool calling
    tools?: Tool[];
    tool_choice?: 'auto' | 'none' | { type: 'function'; name: string };
    parallel_tool_calls?: boolean;

    // Advanced parameters
    top_p?: number;
    top_k?: number;
    frequency_penalty?: number;
    presence_penalty?: number;
    stop?: string[];
    seed?: number;

    // Reasoning
    reasoning?: {
      effort?: 'minimal' | 'low' | 'medium' | 'high';
      max_tokens?: number;
      exclude?: boolean;
    };

    // Anannas-specific
    provider?: ProviderPreferences;
    modalities?: string[];
    audio?: AudioConfig;
    mcp?: MCPConfig;
    prompt_cache_key?: string;
  };
  ```
</CodeGroup>

### Response Format

The Responses API returns a structured response with an `output` array:

<CodeGroup>
  ```typescript TypeScript theme={null}
  type ResponsesResponse = {
    id: string;
    object: 'response';
    model: string;
    created: number;
    output: ResponsesOutputItem[];
    usage: {
      prompt_tokens: number;
      completion_tokens: number;
      total_tokens: number;
    };
    metadata?: { [key: string]: any };
  };
  ```
</CodeGroup>

**Example Response:**

```json theme={null}
{
  "id": "resp_abc123",
  "object": "response",
  "model": "openai/gpt-5-mini",
  "created": 1693350000,
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "status": "completed",
      "content": [
        {
          "type": "output_text",
          "text": "Hello! How can I help you today?"
        }
      ]
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 8,
    "total_tokens": 18
  }
}
```

### Input Format

The `input` field accepts either a simple string or an array of message objects:

**Simple String Input:**

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "input": "What is the capital of France?"
}
```

**Structured Message Input:**

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "What is the capital of France?"
        }
      ]
    }
  ]
}
```

### Output Format

The `output` array contains one or more output items. Each item can be:

* **Message**: Text response from the model
* **Function Call**: Tool/function invocation
* **Error**: Error information

**Message Output:**

```json theme={null}
{
  "type": "message",
  "role": "assistant",
  "status": "completed",
  "content": [
    {
      "type": "output_text",
      "text": "The capital of France is Paris."
    }
  ]
}
```

**Function Call Output:**

```json theme={null}
{
  "type": "message",
  "role": "assistant",
  "status": "completed",
  "content": [
    {
      "type": "function_call",
      "function_call": {
        "id": "call_abc123",
        "name": "get_weather",
        "arguments": "{\"location\": \"Paris\"}"
      }
    }
  ]
}
```

### Prompt Caching

<Card title="Check Caching Support" icon="book">
  For models that support prompt caching and current pricing, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

Prompt caching allows you to reduce costs and latency by reusing cached prompt prefixes. Anannas supports two caching methods depending on the provider:

#### OpenAI Models (`prompt_cache_key`)

For OpenAI models, use the `prompt_cache_key` parameter:

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "input": "Hello, world!",
  "prompt_cache_key": "my-cache-key-123"
}
```

**Pricing:**

* **Cache reads**: Cached input tokens are billed at **50%** of the original input token price
* **Cache writes**: No additional cost for creating the cache

**How it works:**

1. First request with a `prompt_cache_key` creates the cache
2. Subsequent requests with the same key reuse the cached prefix
3. Cache is automatically managed by the provider

#### Anthropic Models (`cache_control`)

For Anthropic Claude models, use the `cache_control` object in message content:

```json theme={null}
{
  "model": "anthropic/claude-sonnet-4",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Your system instructions here...",
          "cache_control": {
            "type": "ephemeral",
            "ttl": "5m"
          }
        }
      ]
    }
  ]
}
```

**Pricing:**

* **Cache creation**: Cache creation tokens are billed at **1.25x** (125%) of the original input token price
* **Cache reads**: Cached input tokens are billed at **0.1x** (10%) of the original input token price - a **90% discount**

**Anthropic-specific requirements:**

* Maximum of **4 content blocks** can have `cache_control` per request
* Cache expires after **5 minutes** (TTL: `"5m"`)
* `cache_control` must be added to individual content parts within messages
* Only `"ephemeral"` cache type is supported

**Example with system message:**

```json theme={null}
{
  "model": "anthropic/claude-sonnet-4",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "You are a helpful assistant. Always be concise.",
          "cache_control": {
            "type": "ephemeral",
            "ttl": "5m"
          }
        },
        {
          "type": "text",
          "text": "What is 2+2?"
        }
      ]
    }
  ]
}
```

#### Other Providers

<Card title="View Caching Support" icon="book">
  For complete caching support details by provider and model, check [anannas.ai/models](https://anannas.ai/models).
</Card>

* **Grok**: Supports caching similar to OpenAI with cached tokens at **10%** of input price (90% discount)
* **Nebius**: Supports caching with provider-specific pricing
* **TogetherAI**: Supports caching with provider-specific pricing

#### Monitoring Cache Usage

Cache usage is included in the response `usage` object:

```json theme={null}
{
  "usage": {
    "prompt_tokens": 1000,
    "completion_tokens": 500,
    "total_tokens": 1500,
    "cache_read_input_tokens": 800,
    "cache_creation_input_tokens": 200
  }
}
```

* `cache_read_input_tokens`: Number of tokens read from cache (discounted pricing)
* `cache_creation_input_tokens`: Number of tokens used to create the cache (Anthropic only, 1.25x pricing)

### Error Handling

The Anannas API returns standard HTTP status codes and error responses:

```json theme={null}
{
  "error": {
    "message": "Invalid request parameters",
    "type": "invalid_request_error",
    "code": "invalid_parameter"
  }
}
```

Common error codes:

* `400 Bad Request`: Invalid request parameters
* `401 Unauthorized`: Missing or invalid API key
* `429 Too Many Requests`: Rate limit exceeded
* `500 Internal Server Error`: Server error

### Rate Limits

Rate limits are applied per API key. See the [Limits documentation](/API/LIMITS) for details.

### Next Steps

* Learn [basic usage](/API/Responses/BasicUsage) with simple text requests
* Explore [streaming responses](/API/Responses/Streaming) for real-time interactions
* Integrate [tool calling](/API/Responses/ToolCalling) for function execution
* Configure [reasoning capabilities](/Features/reasoning) for advanced problem-solving

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Basic Usage
Source: https://docs.anannas.ai/API/Responses/BasicUsage

Learn the fundamentals of making requests with the Responses API.

This guide covers the basics of using the Anannas Responses API, including simple text requests, conversation management, and handling responses.

<Warning>
  **Stateless API**

  The Anannas Responses API is **stateless**. Each request is independent and no conversation state is stored server-side. You must include the full conversation history in every request to maintain context.
</Warning>

### Simple Text Request

The simplest way to use the Responses API is with a plain text string:

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": "What is the capital of France?",
          "max_output_tokens": 1000,
      },
  )

  result = response.json()
  print(result["output"][0]["content"][0]["text"])
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: 'What is the capital of France?',
      max_output_tokens: 1000,
    }),
  });

  const result = await response.json();
  console.log(result.output[0].content[0].text);
  ```

  ```bash cURL theme={null}
  curl https://api.anannas.ai/api/v1/responses \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer <ANANNAS_API_KEY>" \
    -d '{
      "model": "openai/gpt-5-mini",
      "input": "What is the capital of France?",
      "max_output_tokens": 1000
    }'
  ```
</CodeGroup>

### Structured Message Input

For more control, use structured message format with conversation history:

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": [
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "What is the capital of France?"
                      }
                  ]
              }
          ],
          "max_output_tokens": 1000,
      },
  )

  result = response.json()
  print(result["output"][0]["content"][0]["text"])
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: [
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: 'What is the capital of France?',
            },
          ],
        },
      ],
      max_output_tokens: 1000,
    }),
  });

  const result = await response.json();
  console.log(result.output[0].content[0].text);
  ```
</CodeGroup>

### Multi-Turn Conversations

<Warning>
  **Important: Stateless Design**

  The Anannas Responses API is **stateless** - it does not maintain conversation state between requests. Unlike OpenAI's stateful Responses API, you must include the complete conversation history in every request. Each request is treated as independent.
</Warning>

Since the API is stateless, you must include the full conversation history in each request:

<CodeGroup>
  ```python Python theme={null}
  import requests

  # First request
  first_response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": [
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "What is the capital of France?"
                      }
                  ]
              }
          ],
          "max_output_tokens": 1000,
      },
  )

  first_result = first_response.json()
  assistant_message = first_result["output"][0]

  # Second request - include previous conversation
  second_response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": [
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "What is the capital of France?"
                      }
                  ]
              },
              {
                  "type": "message",
                  "role": "assistant",
                  "status": "completed",
                  "content": assistant_message["content"]
              },
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "What is the population of that city?"
                      }
                  ]
              }
          ],
          "max_output_tokens": 1000,
      },
  )

  second_result = second_response.json()
  print(second_result["output"][0]["content"][0]["text"])
  ```

  ```typescript TypeScript theme={null}
  // First request
  const firstResponse = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: [
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: 'What is the capital of France?',
            },
          ],
        },
      ],
      max_output_tokens: 1000,
    }),
  });

  const firstResult = await firstResponse.json();
  const assistantMessage = firstResult.output[0];

  // Second request - include previous conversation
  const secondResponse = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: [
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: 'What is the capital of France?',
            },
          ],
        },
        {
          type: 'message',
          role: 'assistant',
          status: 'completed',
          content: assistantMessage.content,
        },
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: 'What is the population of that city?',
            },
          ],
        },
      ],
      max_output_tokens: 1000,
    }),
  });

  const secondResult = await secondResponse.json();
  console.log(secondResult.output[0].content[0].text);
  ```
</CodeGroup>

### Using Instructions

You can provide system-level instructions to guide the model's behavior:

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": "Explain quantum computing in simple terms.",
          "instructions": "You are a helpful science teacher. Explain concepts clearly and use analogies when helpful.",
          "max_output_tokens": 2000,
      },
  )

  result = response.json()
  print(result["output"][0]["content"][0]["text"])
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: 'Explain quantum computing in simple terms.',
      instructions: 'You are a helpful science teacher. Explain concepts clearly and use analogies when helpful.',
      max_output_tokens: 2000,
    }),
  });

  const result = await response.json();
  console.log(result.output[0].content[0].text);
  ```
</CodeGroup>

### Handling Responses

The response contains an `output` array with one or more output items. Each item has:

* `type`: The type of output (usually "message")
* `role`: The role of the message (usually "assistant")
* `status`: The status of the output ("completed", "incomplete", etc.)
* `content`: An array of content parts

**Extracting Text Content:**

<CodeGroup>
  ```python Python theme={null}
  result = response.json()

  # Get the first output item
  output_item = result["output"][0]

  # Extract text from content
  for content_part in output_item["content"]:
      if content_part["type"] == "output_text":
          print(content_part["text"])
  ```

  ```typescript TypeScript theme={null}
  const result = await response.json();

  // Get the first output item
  const outputItem = result.output[0];

  // Extract text from content
  for (const contentPart of outputItem.content) {
    if (contentPart.type === 'output_text') {
      console.log(contentPart.text);
    }
  }
  ```
</CodeGroup>

### Error Handling

Always check for errors in the response:

<CodeGroup>
  ```python Python theme={null}
  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": "Hello!",
      },
  )

  if response.status_code != 200:
      error = response.json()
      print(f"Error: {error['error']['message']}")
  else:
      result = response.json()
      print(result["output"][0]["content"][0]["text"])
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: 'Hello!',
    }),
  });

  if (!response.ok) {
    const error = await response.json();
    console.error(`Error: ${error.error.message}`);
  } else {
    const result = await response.json();
    console.log(result.output[0].content[0].text);
  }
  ```
</CodeGroup>

### Best Practices

1. **Always include conversation history**: Since the Anannas API is stateless, include all previous messages in each request. The Anannas API does not remember previous conversations.
2. **Manage state client-side**: You are responsible for maintaining conversation history and including it in each request.
3. **Set appropriate token limits**: Use `max_output_tokens` to control response length
4. **Handle errors gracefully**: Check response status codes and error messages
5. **Use structured input**: Prefer structured message format for better control and multimodal support
6. **Monitor usage**: Check the `usage` field in responses to track token consumption

<Info>
  **Stateless vs Stateful**

  OpenAI's Responses API has both stateless and stateful versions. Anannas implements only the **stateless version**, which means:

  * No server-side conversation state
  * No `previous_response_id` parameter support
  * You must include full conversation history in each request
  * Each request is completely independent
</Info>

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Streaming
Source: https://docs.anannas.ai/API/Responses/Streaming

Receive responses in real-time with streaming.

The Responses API supports streaming responses, allowing you to receive output as it's generated in real-time. This is particularly useful for chat interfaces and applications that need to display responses progressively.

### Enabling Streaming

To enable streaming, set `stream: true` in your request:

<CodeGroup>
  ```python Python theme={null}
  import requests
  import json

  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": "Tell me a story about a robot.",
          "stream": True,
          "max_output_tokens": 2000,
      },
      stream=True,
  )

  for line in response.iter_lines():
      if line:
          line_text = line.decode('utf-8')
          if line_text.startswith('data: '):
              data_str = line_text[6:]
              if data_str == '[DONE]':
                  break
              try:
                  event = json.loads(data_str)
                  if event.get('type') == 'response.output_text.delta':
                      delta = event.get('data', {}).get('delta', '')
                      if delta:
                          print(delta, end='', flush=True)
              except json.JSONDecodeError:
                  pass
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: 'Tell me a story about a robot.',
      stream: true,
      max_output_tokens: 2000,
    }),
  });

  const reader = response.body?.getReader();
  const decoder = new TextDecoder();

  if (reader) {
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value);
      const lines = chunk.split('\n');

      for (const line of lines) {
        if (line.startsWith('data: ')) {
          const dataStr = line.slice(6);
          if (dataStr === '[DONE]') {
            break;
          }
          try {
            const event = JSON.parse(dataStr);
            if (event.type === 'response.output_text.delta') {
              const delta = event.data?.delta || '';
              if (delta) {
                process.stdout.write(delta);
              }
            }
          } catch (e) {
            // Skip invalid JSON
          }
        }
      }
    }
  }
  ```
</CodeGroup>

### Stream Event Types

The streaming API uses Server-Sent Events (SSE) format. Each event has a `type` field indicating the event type:

**Event Types:**

* `response.created`: Initial response creation event
* `response.output_item.added`: New output item added
* `response.output_text.delta`: Text delta (incremental text)
* `response.output_item.done`: Output item completed
* `response.completed`: Response generation completed

**Example Stream Events:**

```
data: {"type":"response.created","data":{"id":"resp_123","created_at":1693350000,"model":"openai/gpt-5-mini"}}

data: {"type":"response.output_item.added","data":{"output_index":0,"item":{"type":"message","role":"assistant"}}}

data: {"type":"response.output_text.delta","data":{"output_index":0,"delta":"Once"}}

data: {"type":"response.output_text.delta","data":{"output_index":0,"delta":" upon"}}

data: {"type":"response.output_text.delta","data":{"output_index":0,"delta":" a"}}

data: {"type":"response.output_text.delta","data":{"output_index":0,"delta":" time"}}

data: {"type":"response.output_item.done","data":{"output_index":0}}

data: {"type":"response.completed","data":{"id":"resp_123"}}

data: [DONE]
```

### Processing Stream Events

Here's a more complete example that handles all event types:

<CodeGroup>
  ```python Python theme={null}
  import requests
  import json

  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": "Explain quantum computing.",
          "stream": True,
          "max_output_tokens": 2000,
      },
      stream=True,
  )

  full_text = ""

  for line in response.iter_lines():
      if line:
          line_text = line.decode('utf-8')
          
          # Skip heartbeat messages
          if line_text.startswith(':ANANNAS PROCESSING'):
              continue
              
          if line_text.startswith('data: '):
              data_str = line_text[6:]
              
              if data_str == '[DONE]':
                  print("\n\nStream completed.")
                  break
                  
              try:
                  event = json.loads(data_str)
                  event_type = event.get('type')
                  event_data = event.get('data', {})
                  
                  if event_type == 'response.created':
                      print(f"Response ID: {event_data.get('id')}")
                      print(f"Model: {event_data.get('model')}\n")
                      
                  elif event_type == 'response.output_text.delta':
                      delta = event_data.get('delta', '')
                      if delta:
                          full_text += delta
                          print(delta, end='', flush=True)
                          
                  elif event_type == 'response.output_item.done':
                      print("\n\nOutput item completed.")
                      
                  elif event_type == 'response.completed':
                      print("\n\nResponse generation completed.")
                      
              except json.JSONDecodeError:
                  pass

  print(f"\n\nFull text: {full_text}")
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: 'Explain quantum computing.',
      stream: true,
      max_output_tokens: 2000,
    }),
  });

  const reader = response.body?.getReader();
  const decoder = new TextDecoder();
  let fullText = '';

  if (reader) {
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value);
      const lines = chunk.split('\n');

      for (const line of lines) {
        // Skip heartbeat messages
        if (line.startsWith(':ANANNAS PROCESSING')) {
          continue;
        }

        if (line.startsWith('data: ')) {
          const dataStr = line.slice(6);

          if (dataStr === '[DONE]') {
            console.log('\n\nStream completed.');
            break;
          }

          try {
            const event = JSON.parse(dataStr);
            const eventType = event.type;
            const eventData = event.data || {};

            if (eventType === 'response.created') {
              console.log(`Response ID: ${eventData.id}`);
              console.log(`Model: ${eventData.model}\n`);
            } else if (eventType === 'response.output_text.delta') {
              const delta = eventData.delta || '';
              if (delta) {
                fullText += delta;
                process.stdout.write(delta);
              }
            } else if (eventType === 'response.output_item.done') {
              console.log('\n\nOutput item completed.');
            } else if (eventType === 'response.completed') {
              console.log('\n\nResponse generation completed.');
            }
          } catch (e) {
            // Skip invalid JSON
          }
        }
      }
    }
  }

  console.log(`\n\nFull text: ${fullText}`);
  ```
</CodeGroup>

### Heartbeat Messages

The streaming API sends heartbeat messages (`:ANANNAS PROCESSING`) to keep the connection alive. These should be ignored when processing events.

### Error Handling in Streams

Errors in streaming responses are sent as events:

```json theme={null}
{
  "error": {
    "message": "Rate limit exceeded",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded"
  }
}
```

Always check for error events in your stream processing:

<CodeGroup>
  ```python Python theme={null}
  for line in response.iter_lines():
      if line:
          line_text = line.decode('utf-8')
          if line_text.startswith('data: '):
              data_str = line_text[6:]
              try:
                  event = json.loads(data_str)
                  
                  # Check for errors
                  if 'error' in event:
                      print(f"Error: {event['error']['message']}")
                      break
                      
                  # Process normal events
                  # ...
              except json.JSONDecodeError:
                  pass
  ```

  ```typescript TypeScript theme={null}
  try {
    const event = JSON.parse(dataStr);
    
    // Check for errors
    if (event.error) {
      console.error(`Error: ${event.error.message}`);
      break;
    }
    
    // Process normal events
    // ...
  } catch (e) {
    // Handle parse errors
  }
  ```
</CodeGroup>

### Best Practices

1. **Handle heartbeats**: Ignore `:ANANNAS PROCESSING` messages
2. **Check for \[DONE]**: Stop processing when you receive `[DONE]`
3. **Error handling**: Always check for error events in the stream
4. **Buffer management**: For long streams, consider buffering and processing in chunks
5. **Connection management**: Handle connection drops gracefully and implement retry logic

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Tool Calling
Source: https://docs.anannas.ai/API/Responses/ToolCalling

Integrate function calling with the Responses API.

The Responses API supports comprehensive tool calling capabilities, allowing models to call functions, execute tools in parallel, and handle complex multi-step workflows.

<Note>
  **Stateless API**

  Remember that this API is stateless. When handling multi-turn tool calls, you must include the complete conversation history (including previous tool calls and results) in each request.
</Note>

### Basic Tool Definition

Define tools using the OpenAI function calling format:

<CodeGroup>
  ```python Python theme={null}
  import requests
  import json

  weather_tool = {
      "type": "function",
      "name": "get_weather",
      "description": "Get the current weather in a location",
      "parameters": {
          "type": "object",
          "properties": {
              "location": {
                  "type": "string",
                  "description": "The city and state, e.g. San Francisco, CA"
              },
              "unit": {
                  "type": "string",
                  "enum": ["celsius", "fahrenheit"]
              }
          },
          "required": ["location"]
      }
  }

  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": [
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "What is the weather in San Francisco?"
                      }
                  ]
              }
          ],
          "tools": [weather_tool],
          "tool_choice": "auto",
          "max_output_tokens": 9000,
      },
  )

  result = response.json()
  print(json.dumps(result, indent=2))
  ```

  ```typescript TypeScript theme={null}
  const weatherTool = {
    type: 'function' as const,
    name: 'get_weather',
    description: 'Get the current weather in a location',
    parameters: {
      type: 'object',
      properties: {
        location: {
          type: 'string',
          description: 'The city and state, e.g. San Francisco, CA',
        },
        unit: {
          type: 'string',
          enum: ['celsius', 'fahrenheit'],
        },
      },
      required: ['location'],
    },
  };

  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: [
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: 'What is the weather in San Francisco?',
            },
          ],
        },
      ],
      tools: [weatherTool],
      tool_choice: 'auto',
      max_output_tokens: 9000,
    }),
  });

  const result = await response.json();
  console.log(JSON.stringify(result, null, 2));
  ```
</CodeGroup>

### Tool Choice Options

Control when and how tools are called using the `tool_choice` parameter:

| Tool Choice                                 | Description                                   |
| ------------------------------------------- | --------------------------------------------- |
| `"auto"`                                    | Model decides whether to call tools (default) |
| `"none"`                                    | Model will not call any tools                 |
| `{"type": "function", "name": "tool_name"}` | Force specific tool call                      |

**Force Specific Tool:**

<CodeGroup>
  ```python Python theme={null}
  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": [
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "Get the weather for New York"
                      }
                  ]
              }
          ],
          "tools": [weather_tool],
          "tool_choice": {
              "type": "function",
              "name": "get_weather"
          },
          "max_output_tokens": 9000,
      },
  )
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: [
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: 'Get the weather for New York',
            },
          ],
        },
      ],
      tools: [weatherTool],
      tool_choice: {
        type: 'function',
        name: 'get_weather',
      },
      max_output_tokens: 9000,
    }),
  });
  ```
</CodeGroup>

### Handling Tool Calls

When a model calls a tool, the response contains function call information in the output:

<CodeGroup>
  ```python Python theme={null}
  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": [
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "What is the weather in Paris?"
                      }
                  ]
              }
          ],
          "tools": [weather_tool],
          "tool_choice": "auto",
          "max_output_tokens": 9000,
      },
  )

  result = response.json()

  # Extract function calls from output
  for output_item in result["output"]:
      for content_part in output_item.get("content", []):
          if content_part.get("type") == "function_call":
              func_call = content_part["function_call"]
              print(f"Function: {func_call['name']}")
              print(f"Arguments: {func_call['arguments']}")
              
              # Execute the function
              args = json.loads(func_call["arguments"])
              # ... execute your function with args ...
              
              # Then make another request with the tool result
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: [
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: 'What is the weather in Paris?',
            },
          ],
        },
      ],
      tools: [weatherTool],
      tool_choice: 'auto',
      max_output_tokens: 9000,
    }),
  });

  const result = await response.json();

  // Extract function calls from output
  for (const outputItem of result.output) {
    for (const contentPart of outputItem.content || []) {
      if (contentPart.type === 'function_call') {
        const funcCall = contentPart.function_call;
        console.log(`Function: ${funcCall.name}`);
        console.log(`Arguments: ${funcCall.arguments}`);
        
        // Execute the function
        const args = JSON.parse(funcCall.arguments);
        // ... execute your function with args ...
        
        // Then make another request with the tool result
      }
    }
  }
  ```
</CodeGroup>

### Multi-Turn Tool Calling

<Warning>
  **Stateless Design**

  Since the Anannas API is stateless, you must include the complete conversation history in each request, including all previous tool calls and their results. The Anannas API does not remember previous tool interactions.
</Warning>

For multi-turn conversations with tool calls, include the tool results in subsequent requests:

<CodeGroup>
  ```python Python theme={null}
  # First request - model calls tool
  first_response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": [
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "What's the weather in Boston? Then recommend what to wear."
                      }
                  ]
              }
          ],
          "tools": [weather_tool],
          "max_output_tokens": 9000,
      },
  )

  first_result = first_response.json()

  # Extract function call
  function_call = None
  for output_item in first_result["output"]:
      for content_part in output_item.get("content", []):
          if content_part.get("type") == "function_call":
              function_call = content_part["function_call"]
              break

  if function_call:
      # Execute the function
      args = json.loads(function_call["arguments"])
      # Simulate weather API call
      weather_result = {"temperature": 45, "condition": "rainy", "humidity": 85}
      
      # Second request - include tool result
      second_response = requests.post(
          "https://api.anannas.ai/api/v1/responses",
          headers={
              "Authorization": "Bearer <ANANNAS_API_KEY>",
              "Content-Type": "application/json",
          },
          json={
              "model": "openai/gpt-5-mini",
              "input": [
                  {
                      "type": "message",
                      "role": "user",
                      "content": [
                          {
                              "type": "input_text",
                              "text": "What's the weather in Boston? Then recommend what to wear."
                          }
                      ]
                  },
                  {
                      "type": "message",
                      "role": "assistant",
                      "status": "completed",
                      "content": first_result["output"][0]["content"]
                  },
                  {
                      "type": "message",
                      "role": "tool",
                      "content": [
                          {
                              "type": "tool_result",
                              "tool_call_id": function_call["id"],
                              "content": json.dumps(weather_result)
                          }
                      ]
                  }
              ],
              "tools": [weather_tool],
              "max_output_tokens": 9000,
          },
      )
      
      second_result = second_response.json()
      print(second_result["output"][0]["content"][0]["text"])
  ```

  ```typescript TypeScript theme={null}
  // First request - model calls tool
  const firstResponse = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: [
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: "What's the weather in Boston? Then recommend what to wear.",
            },
          ],
        },
      ],
      tools: [weatherTool],
      max_output_tokens: 9000,
    }),
  });

  const firstResult = await firstResponse.json();

  // Extract function call
  let functionCall = null;
  for (const outputItem of firstResult.output) {
    for (const contentPart of outputItem.content || []) {
      if (contentPart.type === 'function_call') {
        functionCall = contentPart.function_call;
        break;
      }
    }
  }

  if (functionCall) {
    // Execute the function
    const args = JSON.parse(functionCall.arguments);
    // Simulate weather API call
    const weatherResult = { temperature: 45, condition: 'rainy', humidity: 85 };
    
    // Second request - include tool result
    const secondResponse = await fetch('https://api.anannas.ai/api/v1/responses', {
      method: 'POST',
      headers: {
        'Authorization': 'Bearer <ANANNAS_API_KEY>',
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model: 'openai/gpt-5-mini',
        input: [
          {
            type: 'message',
            role: 'user',
            content: [
              {
                type: 'input_text',
                text: "What's the weather in Boston? Then recommend what to wear.",
              },
            ],
          },
          {
            type: 'message',
            role: 'assistant',
            status: 'completed',
            content: firstResult.output[0].content,
          },
          {
            type: 'message',
            role: 'tool',
            content: [
              {
                type: 'tool_result',
                tool_call_id: functionCall.id,
                content: JSON.stringify(weatherResult),
              },
            ],
          },
        ],
        tools: [weatherTool],
        max_output_tokens: 9000,
      }),
    });
    
    const secondResult = await secondResponse.json();
    console.log(secondResult.output[0].content[0].text);
  }
  ```
</CodeGroup>

### Parallel Tool Calls

Enable parallel tool execution with `parallel_tool_calls`:

<CodeGroup>
  ```python Python theme={null}
  response = requests.post(
      "https://api.anannas.ai/api/v1/responses",
      headers={
          "Authorization": "Bearer <ANANNAS_API_KEY>",
          "Content-Type": "application/json",
      },
      json={
          "model": "openai/gpt-5-mini",
          "input": [
              {
                  "type": "message",
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "Get the weather for both New York and London"
                      }
                  ]
              }
          ],
          "tools": [weather_tool],
          "parallel_tool_calls": True,
          "max_output_tokens": 9000,
      },
  )
  ```

  ```typescript TypeScript theme={null}
  const response = await fetch('https://api.anannas.ai/api/v1/responses', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <ANANNAS_API_KEY>',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5-mini',
      input: [
        {
          type: 'message',
          role: 'user',
          content: [
            {
              type: 'input_text',
              text: 'Get the weather for both New York and London',
            },
          ],
        },
      ],
      tools: [weatherTool],
      parallel_tool_calls: true,
      max_output_tokens: 9000,
    }),
  });
  ```
</CodeGroup>

### Best Practices

1. **Clear tool descriptions**: Provide detailed descriptions for better tool selection
2. **Validate tool results**: Always validate and sanitize tool execution results
3. **Handle errors gracefully**: Implement error handling for tool execution failures
4. **Use parallel calls**: Enable `parallel_tool_calls` when multiple independent tools can run simultaneously
5. **Tool result format**: Return tool results as JSON strings for consistency

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Streaming
Source: https://docs.anannas.ai/API/Streaming

Server-Sent Events streaming implementation

## Overview

Anannas supports streaming responses via Server-Sent Events (SSE). Enable streaming by setting `stream: true` in your request. The Anannas API returns incremental text deltas as tokens are generated, enabling real-time chat interfaces.

## Enabling Streaming

Set the `stream` parameter to `true`:

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "messages": [
    {"role": "user", "content": "Explain quantum computing"}
  ],
  "stream": true,
  "max_tokens": 500
}
```

## Response Format

Streaming responses use Server-Sent Events (SSE) with `Content-Type: text/event-stream`. Each event contains a JSON object following the OpenAI streaming format.

### Event Structure

Each SSE event has this format:

```
data: <json_object>

```

The JSON object structure:

```typescript theme={null}
type StreamChunk = {
  id: string;
  object: "chat.completion.chunk";
  created: number;
  model: string;
  choices: Array<{
    index: number;
    delta: {
      role?: "assistant";
      content?: string;
      tool_calls?: ToolCall[];
    };
    finish_reason?: "stop" | "length" | "tool_calls" | "content_filter" | null;
  }>;
};
```

### Example Stream

```
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{"content":"Quantum"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{"content":" computing"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]
```

## Implementation Examples

### Python

```python theme={null}
import requests
import json

response = requests.post(
  "https://api.anannas.ai/v1/chat/completions",
  headers={
    "Authorization": "Bearer <ANANNAS_API_KEY>",
    "Content-Type": "application/json",
  },
  json={
    "model": "openai/gpt-5-mini",
    "messages": [{"role": "user", "content": "Count to 5"}],
    "stream": True
  },
  stream=True
)

for line in response.iter_lines():
  if line:
    line = line.decode('utf-8')
    if line.startswith('data: '):
      data = line[6:]  # Remove 'data: ' prefix
      if data == '[DONE]':
        break
      try:
        chunk = json.loads(data)
        if chunk['choices'][0]['delta'].get('content'):
          print(chunk['choices'][0]['delta']['content'], end='', flush=True)
      except json.JSONDecodeError:
        pass
```

### TypeScript/JavaScript

```typescript theme={null}
const response = await fetch("https://api.anannas.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer <ANANNAS_API_KEY>",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "openai/gpt-5-mini",
    messages: [{ role: "user", content: "Count to 5" }],
    stream: true,
  }),
});

const reader = response.body?.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader!.read();
  if (done) break;

  const chunk = decoder.decode(value);
  const lines = chunk.split('\n');

  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const data = line.slice(6);
      if (data === '[DONE]') break;

      try {
        const json = JSON.parse(data);
        const content = json.choices[0]?.delta?.content;
        if (content) {
          process.stdout.write(content);
        }
      } catch (e) {
        // Ignore parse errors
      }
    }
  }
}
```

### Using OpenAI SDK

The OpenAI SDK handles streaming automatically:

```python theme={null}
from openai import OpenAI

client = OpenAI(
  base_url="https://api.anannas.ai/v1",
  api_key="<ANANNAS_API_KEY>"
)

stream = client.chat.completions.create(
  model="openai/gpt-5-mini",
  messages=[{"role": "user", "content": "Count to 5"}],
  stream=True
)

for chunk in stream:
  if chunk.choices[0].delta.content:
    print(chunk.choices[0].delta.content, end='', flush=True)
```

## SSE Comments

The stream may include comment lines for keep-alive:

```
: ANANNAS
```

These can be ignored per the [SSE specification](https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation). They serve as connection keep-alive signals.

## Cancellation

Cancel streaming requests by closing the connection. Billing stops immediately for supported providers when the connection is closed.

### Python

```python theme={null}
# Close the response stream
response.close()
```

### TypeScript

```typescript theme={null}
const controller = new AbortController();

fetch(url, {
  signal: controller.signal,
  // ... other options
});

// Cancel
controller.abort();
```

## Tool Calls in Streams

Tool calls are streamed incrementally. The `delta.tool_calls` array contains partial tool call information:

```json theme={null}
{
  "choices": [{
    "delta": {
      "tool_calls": [{
        "index": 0,
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\":\""
        }
      }]
    }
  }]
}
```

Accumulate `arguments` strings to reconstruct the complete JSON.

## Error Handling

Errors in streaming responses are sent as regular SSE events:

```
data: {"error": {"message": "Rate limit exceeded", "type": "rate_limit_error"}}
```

Handle errors by checking for the `error` field in parsed JSON objects.

## Finish Reasons

The final chunk includes `finish_reason`:

* `stop`: Natural completion or stop sequence
* `length`: Reached `max_tokens`
* `tool_calls`: Model requested tool execution
* `content_filter`: Content filtered by safety systems
* `null`: Stream incomplete

## Usage Tracking

Streaming responses include usage information in the final chunk or a separate event. Accumulate token counts across chunks for accurate usage tracking.

## Best Practices

1. **Buffer Management**: Accumulate deltas client-side for display
2. **Error Recovery**: Implement retry logic for network failures
3. **Connection Management**: Handle connection drops gracefully
4. **Token Counting**: Track usage from final chunk or usage event
5. **Cancellation**: Always cancel streams when user navigates away

## See Also

* [API Overview](/API/Overview) - Complete request/response schemas
* [Parameters](/API/Parameters) - All available parameters


# Errors
Source: https://docs.anannas.ai/API/errors

Error handling and response codes

## Error Response Format

All errors return a JSON object with this structure:

```typescript theme={null}
type ErrorResponse = {
  error: {
    message: string;
    type: string;
    code?: string;
    metadata?: Record<string, unknown>;
  };
};
```

The HTTP status code matches the error type. Request validation errors return non-200 status codes. Errors occurring during generation may be returned in the response body or as SSE events (for streaming).

## HTTP Status Codes

### 400 Bad Request

Invalid request parameters, malformed JSON, or missing required fields.

```json theme={null}
{
  "error": {
    "message": "Invalid request: model is required",
    "type": "invalid_request_error"
  }
}
```

Common causes:

* Missing required fields (`model`, `messages`)
* Invalid parameter values (e.g., `temperature` > 2.0)
* Malformed JSON
* Invalid message format

### 401 Unauthorized

Invalid, expired, or missing API key.

```json theme={null}
{
  "error": {
    "message": "Authentication required",
    "type": "authentication_error"
  }
}
```

### 402 Payment Required

Insufficient credits or account balance.

```json theme={null}
{
  "error": {
    "message": "Insufficient credits",
    "type": "insufficient_quota_error"
  }
}
```

### 403 Forbidden

Content moderation failure. Input was flagged by safety systems.

```json theme={null}
{
  "error": {
    "message": "Content flagged by moderation",
    "type": "content_filter_error",
    "metadata": {
      "reasons": ["violence", "harassment"],
      "flagged_input": "...",
      "provider_name": "openai",
      "model_slug": "gpt-5-mini"
    }
  }
}
```

### 408 Request Timeout

Request exceeded timeout limit.

```json theme={null}
{
  "error": {
    "message": "Request timeout",
    "type": "timeout_error"
  }
}
```

### 429 Too Many Requests

Rate limit exceeded.

```json theme={null}
{
  "error": {
    "message": "Rate limit exceeded",
    "type": "rate_limit_error"
  }
}
```

Check rate limits via `GET /v1/key`. See [Limits documentation](/API/LIMITS).

### 502 Bad Gateway

Model provider error or invalid response.

```json theme={null}
{
  "error": {
    "message": "Provider error",
    "type": "provider_error",
    "metadata": {
      "provider_name": "openai",
      "raw": {...}
    }
  }
}
```

### 503 Service Unavailable

No available provider meets routing requirements.

```json theme={null}
{
  "error": {
    "message": "No available provider",
    "type": "service_unavailable_error"
  }
}
```

## Error Types

### `invalid_request_error`

Request validation failed. Check parameter types and required fields.

### `authentication_error`

API key authentication failed. Verify key is valid and active.

### `insufficient_quota_error`

Account lacks sufficient credits. Add credits to continue.

### `rate_limit_error`

Request rate exceeds limits. Implement exponential backoff.

### `content_filter_error`

Input flagged by content moderation. Review input content.

### `timeout_error`

Request exceeded timeout. Retry with shorter context or simpler request.

### `provider_error`

Model provider returned an error. May be transient - retry request.

### `service_unavailable_error`

No providers available matching routing requirements. Adjust `provider` preferences or retry later.

## Error Metadata

Some errors include metadata for debugging:

### Moderation Errors

```typescript theme={null}
type ModerationErrorMetadata = {
  reasons: string[];        // Flag categories
  flagged_input: string;     // Flagged content
  provider_name: string;     // Provider that flagged
  model_slug: string;        // Model identifier
};
```

### Provider Errors

```typescript theme={null}
type ProviderErrorMetadata = {
  provider_name: string;     // Provider identifier
  raw: unknown;              // Raw provider response
};
```

## Error Handling Examples

### Python

```python theme={null}
import requests

try:
  response = requests.post(
    "https://api.anannas.ai/v1/chat/completions",
    headers={"Authorization": f"Bearer {api_key}"},
    json={"model": "openai/gpt-5-mini", "messages": [...]}
  )
  response.raise_for_status()
  data = response.json()
except requests.HTTPError as e:
  if e.response.status_code == 401:
    print("Invalid API key")
  elif e.response.status_code == 402:
    print("Insufficient credits")
  elif e.response.status_code == 429:
    print("Rate limit exceeded - retry after delay")
  else:
    error_data = e.response.json()
    print(f"Error: {error_data['error']['message']}")
```

### TypeScript

```typescript theme={null}
try {
  const response = await fetch("https://api.anannas.ai/v1/chat/completions", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "openai/gpt-5-mini",
      messages: [...]
    }),
  });

  if (!response.ok) {
    const error = await response.json();
    switch (response.status) {
      case 401:
        console.error("Invalid API key");
        break;
      case 402:
        console.error("Insufficient credits");
        break;
      case 429:
        console.error("Rate limit exceeded");
        break;
      default:
        console.error(error.error.message);
    }
    return;
  }

  const data = await response.json();
} catch (error) {
  console.error("Request failed:", error);
}
```

## Retry Logic

Implement exponential backoff for transient errors:

```python theme={null}
import time
import random

def retry_with_backoff(func, max_retries=3):
  for attempt in range(max_retries):
    try:
      return func()
    except requests.HTTPError as e:
      if e.response.status_code in [429, 502, 503]:
        wait = (2 ** attempt) + random.uniform(0, 1)
        time.sleep(wait)
        continue
      raise
  raise Exception("Max retries exceeded")
```

## Streaming Errors

In streaming responses, errors are sent as SSE events:

```
data: {"error": {"message": "Rate limit exceeded", "type": "rate_limit_error"}}
```

Parse and handle errors in your SSE event loop.

## Best Practices

1. **Check Status Codes**: Always verify HTTP status before parsing JSON
2. **Handle 429**: Implement exponential backoff for rate limits
3. **Log Metadata**: Include error metadata in logs for debugging
4. **User-Friendly Messages**: Translate technical errors to user-friendly messages
5. **Retry Transient Errors**: Retry 502, 503, and 408 errors
6. **Monitor Credits**: Track 402 errors and alert when credits are low

## See Also

* [Authentication](/API/Authentication) - API key management
* [Limits](/API/LIMITS) - Rate limits and quotas


# Community
Source: https://docs.anannas.ai/Community/overview

Community resources and integrations

## Integrations

<CardGroup>
  <Card title="Langfuse" icon="chart-line" href="https://langfuse.com/integrations/gateways/anannas">
    LLM observability and tracing platform integration for dual-layer visibility

    **Status:** Live

    Track gateway metrics with Anannas while capturing application traces with Langfuse; full observability from model routing to production execution.
  </Card>
</CardGroup>

***

## What's Coming

We're working on exciting community features and integrations that will make it easier to build with Anannas:

<CardGroup>
  <Card title="Framework Integrations" icon="plug">
    Seamless integration with popular AI frameworks and SDKs

    <Note>**Status:** Coming Soon</Note>
  </Card>

  <Card title="Developer Tools" icon="code">
    Integration with coding assistants and development environments

    <Note>**Status:** Coming Soon</Note>
  </Card>

  <Card title="Community Resources" icon="book">
    Guides, tutorials, and best practices from the community

    <Note>**Status:** Coming Soon</Note>
  </Card>

  <Card title="Collaboration" icon="users">
    Connect with other developers and share your projects

    <Note>**Status:** Coming Soon</Note>
  </Card>

  <Card title="Pipecat Integration" icon="microphone" href="https://github.com/pipecat-ai/docs/pull/401">
    Voice and multimodal AI framework integration for conversational applications

    <Note>**Status:** In Progress</Note>
  </Card>
</CardGroup>

***

<Tip>
  **Have ideas for community features?**

  We'd love to hear from you! Reach out to us on [Discord](https://discord.gg/dJAfsWNm2D) to help shape our community offerings.
</Tip>

<Check>
  **Stay Updated**

  Follow our documentation for updates as we roll out these community features and integrations.
</Check>


# FAQs
Source: https://docs.anannas.ai/FAQs

Common questions about Anannas - The unified AI API gateway

## **Getting Started**

<AccordionGroup>
  <Accordion title="Why should I use Anannas?">
    Anannas is a unified API gateway that provides access to multiple LLM providers (OpenAI, Anthropic, Google, Meta, Mistral, and more) through a single interface. Key benefits include:

    * **Unified API**: One endpoint for all models with consistent authentication and response formats
    * **Intelligent Routing**: Automatic load balancing and provider selection based on price, latency, or throughput
    * **Minimal Overhead**: Sub-20ms latency with comprehensive monitoring and analytics
    * **Cost Optimization**: Route requests to the most cost-effective providers automatically
    * **Fallback Support**: Automatic failover when preferred providers are unavailable
    * **Developer-First**: Built for developers with structured outputs, tool calling, and multimodal support
  </Accordion>

  <Accordion title="How do I get started with Anannas?">
    Getting started with Anannas is simple:

    1. **Register**: Create an account at [anannas.ai](https://anannas.ai)
    2. **Generate API Key**: Create an API key from your dashboard
    3. **Make Your First Request**: Use our OpenAI-compatible endpoints to send requests

    ```bash theme={null}
    curl -X POST "https://api.anannas.ai/v1/chat/completions" \
      -H "Authorization: Bearer YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "openai/gpt-5-mini",
        "messages": [{"role": "user", "content": "Hello Anannas!"}]
      }'
    ```

    Check our [Quick Start Guide](/quickstart) for detailed examples.
  </Accordion>

  <Accordion title="How do I get support?">
    We provide multiple support channels:

    * **Community**: Join our [Telegram](https://t.me/+8KtRNZLaa_M4NDA1) for community support and discussions
    * **Email**: Contact us at [contact@anannas.ai](mailto:contact@anannas.ai) for technical support
    * **Documentation**: Comprehensive guides and API references in our docs
    * **Status Page**: Check system status and uptime at our status page

    For enterprise customers, we offer dedicated support with SLA guarantees.
  </Accordion>
</AccordionGroup>

## **Models and Providers**

<AccordionGroup>
  <Accordion title="What LLM models does Anannas support?">
    Anannas supports a comprehensive range of models from major AI providers:

    **Models:**

    * **OpenAI**: GPT-5, GPT-4o-mini, GPT-3.5-turbo, and more
    * **Anthropic**: Claude 4.5 Sonnet, Claude 3 Opus, Claude 3 Haiku
    * **xAI**: Grok-1, Grok-2 (via direct integration)
    * **Meta**: Llama 3.1, Llama 3.3 series (8B, 70B, 405B)
    * **Google**: Gemini Pro, Gemini Flash, PaLM models
    * **Mistral**: Mistral 7B, Mixtral 8x7B, Codestral
    * **Cohere**: Command, Command Light, Aya models
    * **Alibaba**: Qwen series (Qwen 2.5, Qwen 2.7B, 72B)
    * **DeepSeek**: DeepSeek Coder, DeepSeek Chat models
    * Many more models...

    View the complete list at our [Models page](https://anannas.ai/models).
  </Accordion>

  <Accordion title="How does provider auto-routing work?">
    Anannas includes intelligent routing that automatically selects the best provider for your requests:

    **Routing Strategies:**

    * **Price-based**: Routes to the most cost-effective provider
    * **Latency-based**: Selects providers with lowest response times
    * **Throughput-based**: Chooses providers with highest capacity
    * **Load-balanced**: Distributes requests evenly with price weighting

    **Provider Preferences:**

    ```json theme={null}
    {
      "model": "gpt-4o",
      "provider": {
        "sort": "price",
        "only": ["openai", "anthropic"],
        "ignore": ["google"],
      }
    }
    ```

    **Fallback Support:**

    * Automatic failover when primary providers are unavailable
    * Zero-downtime switching between providers
    * Real-time provider health monitoring
  </Accordion>

  <Accordion title="How frequently are new models added?">
    We continuously expand our model catalog:

    * **Weekly Updates**: New model releases are typically available within days
    * **Partnership Access**: Early access to models through provider partnerships
    * **Community Requests**: Submit model requests via our support channels
  </Accordion>
</AccordionGroup>

## **API and Integration**

<AccordionGroup>
  <Accordion title="What API formats does Anannas support?">
    Anannas provides multiple API formats for maximum compatibility:

    **Completely OpenAI-Compatible:**

    ```bash theme={null}
    POST /v1/chat/completions
    ```

    All endpoints support streaming responses and maintain consistent authentication.
  </Accordion>

  <Accordion title="Does Anannas support streaming responses?">
    Yes! Anannas supports real-time streaming for all compatible models:

    ```bash theme={null}
    curl -X POST "https://api.anannas.ai/v1/chat/completions" \
      -H "Authorization: Bearer YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "gpt-4o",
        "messages": [{"role": "user", "content": "Tell me a story"}],
        "stream": true
      }'
    ```

    Streaming works across all providers with automatic provider selection and fallback support.
  </Accordion>

  <Accordion title="What authentication methods are supported?">
    Anannas uses API key authentication:

    **API Key Authentication:**

    ```bash theme={null}
    Authorization: Bearer YOUR_API_KEY
    ```

    **Key Management:**

    * Generate multiple API keys for different applications
    * Set usage limits and restrictions per key
    * Rotate keys without downtime
    * Monitor key usage and analytics

    **Security Features:**

    * Rate limiting per API key
    * IP allowlisting (enterprise)
    * Request signing (coming soon)
    * Audit logs for all API usage
  </Accordion>

  <Accordion title="How do I handle errors and retries?">
    Anannas provides comprehensive error handling:

    **Standard Error Format:**

    ```json theme={null}
    {
      "error": {
        "type": "invalid_request_error",
        "message": "Model not found",
        "code": "model_not_found"
      }
    }
    ```

    **Retry Logic:**

    * Automatic retries with exponential backoff
    * Provider fallback on failures
    * Circuit breaker for unhealthy providers
    * Detailed error codes for programmatic handling

    **Error Categories:**

    * `authentication_error`: Invalid API key
    * `rate_limit_error`: Rate limit exceeded
    * `insufficient_quota`: Credits exhausted
    * `model_not_found`: Unsupported model
    * `provider_error`: Provider-specific issues
  </Accordion>
</AccordionGroup>

## **Pricing and Billing**

<AccordionGroup>
  <Accordion title="How does billing work on Anannas?">
    Anannas uses a transparent credit-based billing system:

    **Credit System:**

    * Purchase credits in advance
    * Pay only for actual token usage
    * No monthly subscriptions required
    * Credits never expire

    **Pricing Structure:**

    * Model-specific pricing
    * Different rates for prompt and completion tokens
    * Special pricing for image generation and reasoning tokens
    * Volume discounts for high usage

    **Billing Features:**

    * Real-time usage tracking
    * Detailed usage analytics
    * Invoice generation
    * Multiple payment methods (Stripe)
  </Accordion>

  <Accordion title="What payment methods are accepted?">
    We accept multiple payment methods through Stripe:

    * **Credit Cards**: Visa, Mastercard, American Express
    * **Debit Cards**: All major debit card networks
    * **Digital Wallets**: Apple Pay, Google Pay (where available)
    * **Cryptocurrency**: Bitcoin, Ethereum (coming soon)
    * **UPI**: (coming soon)

    All payments are processed securely through Stripe with PCI compliance.
  </Accordion>

  <Accordion title="Are there any fees for using Anannas?">
    Our pricing is transparent with minimal fees:

    **Credit Purchase Fees:**

    * 5% processing fee on credit purchases
    * Minimum fee of \$0.80 per transaction
    * No fees for API usage beyond model costs

    **BYOK (Bring Your Own Keys):**

    * No additional fees when using your own provider API keys
    * Only pay for Anannas infrastructure and routing features
    * Full control over your provider relationships
  </Accordion>

  <Accordion title="What's the refund policy?">
    Our refund policy is developer-friendly:

    **Credit Refunds:**

    * Unused credits can be refunded within 30 days
    * Full refund for technical issues or service problems
    * Prorated refunds for subscription cancellations

    **Usage Refunds:**

    * We don't charge for any error in inference
    * or for provider-side errors
    * Dispute resolution within 48 hours

    **Enterprise Customers:**

    * Custom refund terms in enterprise agreements
    * SLA-based credit compensation
    * Dedicated support for billing issues

    Contact support for any issues.
  </Accordion>
</AccordionGroup>

## **Features and Capabilities**

<AccordionGroup>
  <Accordion title="What advanced features does Anannas support?">
    Anannas provides comprehensive AI capabilities:

    **Structured Outputs:**

    * JSON schema enforcement
    * Type-safe responses
    * Custom output formats

    **Tool Calling:**

    * Function execution through LLMs
    * Multi-turn tool conversations
    * Custom tool definitions

    **Multimodal Support:**

    * Image input processing
    * PDF document analysis
    * Video content understanding (coming soon)

    **Advanced Routing:**

    * Custom provider preferences
    * Geographic routing
    * Load balancing strategies
    * A/B testing capabilities
  </Accordion>

  <Accordion title="How does Anannas handle rate limits?">
    Our rate limiting system is designed for fairness and reliability:

    **Per-API-Key Limits:**

    * Burst allowance for short spikes
    * Automatic scaling based on usage patterns

    **Provider-Level Limits:**

    * Respects individual provider rate limits
    * Automatic load balancing across providers
    * Queue management for high-demand periods

    **Enterprise Limits:**

    * Custom rate limits based on plan
    * Priority access during high traffic
    * Dedicated capacity allocation
  </Accordion>

  <Accordion title="Does Anannas support organization accounts?">
    Yes! Organization accounts provide team management features:

    **Organization Features:**

    * Team member management
    * Role-based access control
    * Shared billing and usage tracking
    * Organization-wide API keys

    **Roles Available:**

    * **Owner**: Full access and billing control
    * **Admin**: User and billing management
    * **Member**: API access with usage monitoring
    * **Viewer**: Read-only access to analytics

    **Billing Benefits:**

    * Consolidated billing for all team usage
    * Volume discounts across team usage
    * Invoice management for accounting
    * Usage analytics per team member
  </Accordion>
</AccordionGroup>

## **Privacy and Security**

<AccordionGroup>
  <Accordion title="What data does Anannas log and store?">
    We maintain strict privacy standards:

    **Logged Data:**

    * Request metadata (timestamp, model, token counts)
    * API key usage statistics - for dashboard

    **Not Logged:**

    * Request content
    * Response content
    * Personal information

    **Data Retention:**

    * Usage logs: 90 days for analytics
    * No content storage beyond processing time
  </Accordion>

  <Accordion title="What are Zero Data Retention (ZDR) providers?">
    ZDR providers ensure no data persistence:

    **ZDR Features:**

    * No request/response logging
    * In-memory processing only
    * Automatic data deletion after processing
    * Compliance with strict data regulations

    **Use Cases:**

    * Healthcare data processing
    * Financial information handling
    * Legal document analysis
    * Sensitive corporate data

    **ZDR Providers:**

    * Select providers offer ZDR guarantees
    * Audit logs for compliance verification
  </Accordion>
</AccordionGroup>

## **Account and API Management**

<AccordionGroup>
  <Accordion title="How do I manage multiple API keys?">
    Anannas supports multiple API keys for different applications and environments:

    **Key Management Features:**

    * Create as many API keys per account
    * Set custom names for easy identification
    * Monitor usage per key in the dashboard
    * Revoke keys instantly when compromised

    **Best Practices:**

    * Use separate keys for development, staging, and production
    * Name keys descriptively (e.g., "production-web-app", "staging-mobile")
    * Rotate keys regularly for security
    * Monitor usage patterns to detect anomalies

    **Organization Keys:**

    * Organizations can create shared API keys
    * Team members can use organization keys
    * Usage is tracked per organization
    * Admins can manage all organization keys
  </Accordion>

  <Accordion title="How do I monitor my API usage and costs?">
    Comprehensive usage monitoring is available through the dashboard:

    **Real-time Analytics:**

    * Live usage dashboard with request counts
    * Token usage breakdown by model
    * Cost tracking and projections
    * Provider routing statistics

    **Usage Alerts:**

    * Low balance warnings and auto topup triggers
    * High usage notifications
    * Rate limit approaching alerts
    * Monthly usage summaries

    **Detailed Reports:**

    * Daily, weekly, monthly usage reports
    * Model performance comparisons
    * Cost optimization suggestions
    * Export data for accounting

    Access your dashboard at [anannas.ai/dashboard](https://anannas.ai/).
  </Accordion>

  <Accordion title="How do I set up team accounts and organizations?">
    Organization accounts provide team collaboration features:

    **Creating Organizations:**

    1. Go to your dashboard → Organizations
    2. Click "Create Organization"
    3. Set organization name
    4. Invite team members via email

    **Billing Benefits:**

    * Consolidated billing for all team usage
    * Volume discounts across team members
    * Shared credit pool
    * Invoice management for accounting teams

    **Invitation Process:**

    * Send invitations via email
    * Members accept invitations to join
    * Automatic role assignment
    * Usage tracking per member
  </Accordion>
</AccordionGroup>

## **Troubleshooting**

<AccordionGroup>
  <Accordion title="What should I do if my requests are failing?">
    Follow these steps to diagnose and resolve request failures:

    **Immediate Checks:**

    1. Verify API key is valid and active
    2. Check account balance and credits
    3. Confirm model name is correct
    4. Validate request format and parameters

    **Provider Issues:**

    * Check provider status in dashboard
    * Try different models or providers
    * Use provider routing to avoid failed providers
    * Monitor provider-specific error rates

    **Debugging Tools:**

    * Use request/response logging
    * Check error details in dashboard
    * Monitor provider routing decisions
    * Review usage analytics for patterns

    **Getting Help:**

    * Check our status page for outages
    * Contact support with error details
    * Join our [Discord community](https://anannas.ai/discord) for help
    * Review documentation for examples
  </Accordion>
</AccordionGroup>

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Image Input
Source: https://docs.anannas.ai/Features/Multimodality/Images

Send images to vision models through the Anannas API.

Requests with images, to multimodal models, are available via the `/chat/completions` API with a multi-part `messages`parameter. The `image_url` can either be a **URL** or a **base64-encoded image**. You can send multiple images by adding more content entries in the same message.

<Note>
  URLs are preferred for large files since they avoid payload bloat. Use base64 encoding when working with local or private files.
</Note>

Anannas supports both **direct URLs** and **base64-encoded data** for images:

* **URLs**: More efficient for publicly accessible images as they don't require local encoding
* **Base64**: Required for local files or private images that aren't publicly accessible

### Using Image URLs

Here's how to send an image via URL:

<CodeGroup>
  ```python Python theme={null}
  import requests
  import json

  url = "https://api.anannas.ai/v1/chat/completions"
  headers = {
      "Authorization": f"Bearer {API_KEY_REF}",
      "Content-Type": "application/json"
  }

  messages = [
      {
          "role": "user",
          "content": [
              {"type": "text", "text": "What do you see in this image?"},
              {
                  "type": "image_url",
                  "image_url": {
                      "url": "https://upload.wikimedia.org/wikipedia/commons/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
                      "detail": "high"
                  }
              }
          ]
      }
  ]

  payload = {"model": "{{MODEL}}", "messages": messages, "max_tokens": 300}

  response = requests.post(url, headers=headers, json=payload)
  print(response.json())
  ```

  ```typescript Typescript theme={null}
  const response = await fetch('https://api.anannas.ai/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${API_KEY_REF}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: '{{MODEL}}',
      max_tokens: 300,
      messages: [
        {
          role: 'user',
          content: [
            { type: 'text', text: "What do you see in this image?" },
            {
              type: 'image_url',
              image_url: {
                url: 'https://upload.wikimedia.org/wikipedia/commons/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg',
                detail: 'high',
              },
            },
          ],
        },
      ],
    }),
  });

  const data = await response.json();
  console.log(data);
  ```
</CodeGroup>

### Using Base64 Encoded Images (OpenAI only)

For **OpenAI models**, you can send local images as base64:

<CodeGroup>
  ```python Python theme={null}
  import requests, base64

  def encode_image_to_base64(image_path):
      with open(image_path, "rb") as f:
          return base64.b64encode(f.read()).decode("utf-8")

  url = "https://api.anannas.ai/v1/chat/completions"
  headers = {
      "Authorization": f"Bearer {API_KEY_REF}",
      "Content-Type": "application/json"
  }

  base64_image = encode_image_to_base64("path/to/image.jpg")
  data_url = f"data:image/jpeg;base64,{base64_image}"

  messages = [
      {
          "role": "user",
          "content": [
              {"type": "text", "text": "What do you see in this image?"},
              {
                  "type": "image_url",
                  "image_url": {"url": data_url, "detail": "high"}
              }
          ]
      }
  ]

  payload = {"model": "{{MODEL}}", "messages": messages, "max_tokens": 300}

  response = requests.post(url, headers=headers, json=payload)
  print(response.json())
  ```

  ```typescript Typescript theme={null}
  import fs from 'fs/promises';

  async function encodeImageToBase64(path: string): Promise<string> {
    const buffer = await fs.readFile(path);
    return `data:image/jpeg;base64,${buffer.toString('base64')}`;
  }

  const imageData = await encodeImageToBase64('path/to/image.jpg');

  const response = await fetch('https://api.anannas.ai/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${API_KEY_REF}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: '{{MODEL}}',
      max_tokens: 300,
      messages: [
        {
          role: 'user',
          content: [
            { type: 'text', text: "What do you see in this image?" },
            {
              type: 'image_url',
              image_url: { url: imageData, detail: 'high' },
            },
          ],
        },
      ],
    }),
  });

  const data = await response.json();
  console.log(data);
  ```
</CodeGroup>

### Supported image types:

<Card title="Check Vision Model Support" icon="book">
  For models that support image inputs and their capabilities, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

* `image/png`
* `image/jpeg`
* `image/webp`
* `image/gif`

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Multimodal Capabilities
Source: https://docs.anannas.ai/Features/Multimodality/Overview

Send images, PDFs, and audio to Anannas

Anannas supports **multiple input modalities beyond text**, so you can send images, PDFs, and audio files to compatible models through the same `/chat/completions` endpoint.  This enables rich, multimodal interactions with minimal integration overhead.

### Supported Modalities

* **Images** - Send images to vision-enabled models for tasks like analysis, captioning, OCR, and visual Q\&A. Supports: **URL-based images**, **base64-encoded images**\
  [Read More about Image Inputs ->](/Features/Multimodality/Images)

* **PDFs** - Process PDF documents seamlessly. Anannas extracts text and handles both **text-based PDFs** and **scanned files**. Accepted formats: **URL** or **base64-encoded**.\
  [Read More about PDF processing ->](/Features/Multimodality/PDFs)

* **Audio** - Send audio files to models with audio processing capabilities for tasks like transcription, translation, and multimodal analysis. Supports: **base64-encoded audio** (URLs not supported).\
  [Read More about Audio Input ->](/Features/Multimodality/audio)

### Getting Started

All multimodal inputs use `/chat/completions` with the `messages` array.  \
Specify the content type for each input:

1. **Images** -> `image_url`
2. **PDFs** -> `file`
3. **Audio** -> `input_audio`

You can combine multiple inputs in a single request. The maximum number of files depends on the model/provider.

### Model Compatibility

<Card title="Check Multimodal Support" icon="book">
  For models that support images, PDFs, and other modalities, visit [anannas.ai/models](https://anannas.ai/models) to see capabilities by model.
</Card>

1. **Images** -> Supported (URL + base64)
2. **PDFs** -> Supported (URL + base64)
3. **Audio** -> Supported (base64 only)

You can combine multiple modalities in a single request, and the number of files you can send varies by provider and model.

### Input Format Support

**URLs (Recommended for public content)**

1. Images: `https://example.com/image.jpg`
2. PDFs: `https://example.com/document.pdf`
3. Audio: ❌ Not supported (base64 only)

**Base64 Encoding (Required for local/private files)**

1. Images: `data:image/jpeg;base64,{base64_data}`
2. PDFs: `data:application/pdf;base64,{base64_data}`
3. Audio: Base64-encoded with format specification

<Note>
  URLs are preferred for large files since they avoid payload bloat.

  Use base64 encoding when working with local or private files.
</Note>

### FAQs

<AccordionGroup>
  <Accordion title="Can I send both images and PDFs in one request?">
    Yes. You can mix text, images, and PDFs in the same request. The model will process them together.
  </Accordion>

  <Accordion title="What happens if I send audio to a model that doesn't support it?">
    Audio inputs are only processed by models with audio input modality. If you send audio to a model that doesn't support it, the audio will be ignored silently (no error will be thrown).
  </Accordion>

  <Accordion title="Which models support audio?">
    Models with audio input modality support audio processing. Check the [Models page](https://anannas.ai/models) and filter by audio input modality to see which models support audio.
  </Accordion>
</AccordionGroup>

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# PDF Inputs
Source: https://docs.anannas.ai/Features/Multimodality/PDFs

Send PDF files to any model on Anannas

Anannas supports **PDF file processing** through the `/chat/completions` API.  \
PDFs can be sent as **direct URLs** or **base64-encoded data URLs** in the messages array, using the `file` content type. This works on **all models**, even if the model doesn’t natively support PDFs.

<Note>
  URLs are preferred for large files since they avoid payload bloat.

  Use base64 encoding when working with local or private files.
</Note>

### Using PDF URLs

For publicly accessible PDFs, send the file URL directly:

<CodeGroup>
  ```json JSON theme={null}
  import requests

  url = "https://api.anannas.ai/v1/chat/completions"
  headers = {
      "Authorization": f"Bearer {API_KEY_REF}",
      "Content-Type": "application/json",
  }

  messages = [
      {
          "role": "user",
          "content": [
              { "type": "text", "text": "Summarize this PDF." },
              {
                  "type": "file",
                  "file": {
                      "filename": "sample.pdf",
                      "file_data": "https://example.com/sample.pdf",
                  },
              },
          ],
      }
  ]

  payload = { "model": "openai/gpt-5-mini", "messages": messages }

  response = requests.post(url, headers=headers, json=payload)
  print(response.json())
  ```

  ```python Python theme={null}
  const response = await fetch("https://api.anannas.ai/v1/chat/completions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${API_KEY_REF}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "openai/gpt-5-mini",
      messages: [
        {
          role: "user",
          content: [
            { type: "text", text: "Summarize this PDF." },
            {
              type: "file",
              file: {
                filename: "sample.pdf",
                file_data: "https://example.com/sample.pdf",
              },
            },
          ],
        },
      ],
    }),
  });

  const data = await response.json();
  console.log(data);
  ```
</CodeGroup>

### Using Base64-Encoded PDFs

For local or private PDFs, encode to base64 and send as a data URL:

<CodeGroup>
  ```python Python theme={null}
  import base64
  import requests

  def encode_file_to_base64(file_path):
      with open(file_path, "rb") as f:
          return base64.b64encode(f.read()).decode("utf-8")

  pdf_path = "document.pdf"
  base64_pdf = encode_file_to_base64(pdf_path)
  data_url = f"data:application/pdf;base64,{base64_pdf}"

  url = "https://api.anannas.ai/v1/chat/completions"
  headers = {
      "Authorization": f"Bearer {API_KEY_REF}",
      "Content-Type": "application/json",
  }

  messages = [
      {
          "role": "user",
          "content": [
              { "type": "text", "text": "Extract key points from this PDF." },
              {
                  "type": "file",
                  "file": {
                      "filename": "document.pdf",
                      "file_data": data_url,
                  },
              },
          ],
      }
  ]

  payload = { "model": "anthropic/claude-3-5-sonnet-20241022", "messages": messages }

  response = requests.post(url, headers=headers, json=payload)
  print(response.json())
  ```

  ```Typescript theme={null}
  import fs from "fs";

  async function encodeFileToBase64(filePath: string): Promise<string> {
    const buffer = await fs.promises.readFile(filePath);
    return `data:application/pdf;base64,${buffer.toString("base64")}`;
  }

  const base64File = await encodeFileToBase64("document.pdf");

  const response = await fetch("https://api.anannas.ai/v1/chat/completions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${API_KEY_REF}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "anthropic/claude-3-5-sonnet-20241022",
      messages: [
        {
          role: "user",
          content: [
            { type: "text", text: "Extract key points from this PDF." },
            {
              type: "file",
              file: {
                filename: "document.pdf",
                file_data: base64File,
              },
            },
          ],
        },
      ],
    }),
  });

  const data = await response.json();
  console.log(data);
  ```
</CodeGroup>

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Audio Inputs
Source: https://docs.anannas.ai/Features/Multimodality/audio

How to send audio files to Anannas models

Anannas supports sending audio files to compatible models via the API. This guide will show you how to work with audio using Anannas.

<Note>
  Audio files must be **base64-encoded** - direct URLs are not supported for audio content.
</Note>

### Audio Inputs

Requests with audio files to compatible models are available via the `/v1/chat/completions` API with the `input_audio` content type. Audio files must be base64-encoded and include the format specification. Note that only models with audio processing capabilities will handle these requests.

You can search for models that support audio by filtering to audio input modality on our [Models page](https://anannas.ai/models).

### Sending Audio Files

Here's how to send an audio file for processing:

<CodeGroup>
  ```python Python theme={null}
  import requests
  import base64
  from pathlib import Path

  def transcribe_audio(file_path, model_id, max_tokens=4096):
      """Transcribe audio file using Anannas API."""
      
      # Read and encode audio file
      with open(file_path, 'rb') as f:
          audio_data = base64.b64encode(f.read()).decode('utf-8')
      
      # Get file extension for format (e.g., .mp3 -> mp3)
      file_ext = Path(file_path).suffix.lstrip('.').lower()
      
      # Prepare payload
      payload = {
          'model': model_id,
          'messages': [{
              'role': 'user',
              'content': [
                  {
                      'type': 'text',
                      'text': 'Transcribe and summarize this audio.'
                  },
                  {
                      'type': 'input_audio',
                      'input_audio': {
                          'data': audio_data,
                          'format': file_ext
                      }
                  }
              ]
          }],
          'max_tokens': max_tokens
      }
      
      # Make API request
      response = requests.post(
          'https://api.anannas.ai/v1/chat/completions',
          headers={
              'Authorization': f'Bearer {API_KEY_REF}',
              'Content-Type': 'application/json'
          },
          json=payload,
          timeout=60
      )
      
      if response.status_code == 200:
          result = response.json()
          content = result['choices'][0]['message']['content']
          print(content)
          
          # Print token usage if available
          if 'usage' in result:
              print(f"\nTokens used: {result['usage'].get('total_tokens', 'N/A')}")
          return content
      else:
          print(f'Error: {response.status_code} - {response.text}')
          return None

  # Example usage
  transcribe_audio('path/to/your/audio.wav', 'google/gemini-3-flash-preview', max_tokens=20000)
  ```

  ```typescript Typescript theme={null}
  import fs from 'fs/promises';
  import path from 'path';

  async function transcribeAudio(
    filePath: string,
    modelId: string,
    maxTokens: number = 4096
  ): Promise<string | null> {
    // Read and encode audio file
    const buffer = await fs.readFile(filePath);
    const base64Audio = buffer.toString('base64');
    
    // Get file extension for format (e.g., .mp3 -> mp3)
    const fileExt = path.extname(filePath).slice(1).toLowerCase();
    
    // Make API request
    const response = await fetch('https://api.anannas.ai/v1/chat/completions', {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${API_KEY_REF}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model: modelId,
        messages: [
          {
            role: 'user',
            content: [
              {
                type: 'text',
                text: 'Transcribe and summarize this audio.',
              },
              {
                type: 'input_audio',
                input_audio: {
                  data: base64Audio,
                  format: fileExt,
                },
              },
            ],
          },
        ],
        max_tokens: maxTokens,
      }),
    });
    
    if (response.ok) {
      const result = await response.json();
      const content = result.choices[0].message.content;
      console.log(content);
      
      // Print token usage if available
      if (result.usage) {
        console.log(`\nTokens used: ${result.usage.total_tokens || 'N/A'}`);
      }
      return content;
    } else {
      const error = await response.text();
      console.error(`Error: ${response.status} - ${error}`);
      return null;
    }
  }

  // Example usage
  await transcribeAudio('path/to/your/audio.wav', 'google/gemini-3-flash-preview', 20000);
  ```
</CodeGroup>

### Format Detection

The audio format is typically determined from the file extension. Extract the format by removing the leading dot from the file extension (e.g., `.mp3` → `mp3`, `.wav` → `wav`).

### Supported Formats

Supported audio formats vary by provider. Common formats include:

* `wav` - WAV audio
* `mp3` - MP3 audio
* `aiff` - AIFF audio
* `aac` - AAC audio
* `ogg` - OGG Vorbis audio
* `flac` - FLAC audio
* `m4a` - M4A audio
* `pcm16` - PCM16 audio
* `pcm24` - PCM24 audio

<Card title="Check Format Support" icon="book">
  Note: Check your model's documentation to confirm which audio formats it supports. Not all models support all formats. Visit [anannas.ai/models](https://anannas.ai/models) to see capabilities by model.
</Card>

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Structured Outputs
Source: https://docs.anannas.ai/Features/structured-outputs

JSON Schema validation for model responses

## Overview

Structured outputs enforce JSON Schema validation on model responses, ensuring consistent, type-safe outputs. This eliminates parsing errors and hallucinated fields, simplifying downstream integrations.

## Request Format

Use `response_format` with `type: "json_schema"`:

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "messages": [
    {"role": "user", "content": "What's the weather like in London?"}
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "weather",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "City or location name"
          },
          "temperature": {
            "type": "number",
            "description": "Temperature in Celsius"
          },
          "conditions": {
            "type": "string",
            "description": "Weather conditions description"
          }
        },
        "required": ["location", "temperature", "conditions"],
        "additionalProperties": false
      }
    }
  }
}
```

## Response Format

The model returns JSON matching your schema:

```json theme={null}
{
  "choices": [{
    "message": {
      "content": "{\"location\": \"London\", \"temperature\": 18, \"conditions\": \"Partly cloudy with light drizzle\"}"
    }
  }]
}
```

Parse the content as JSON:

```python theme={null}
import json

response = client.chat.completions.create(...)
content = response.choices[0].message.content
weather_data = json.loads(content)
```

## Schema Configuration

### `name`

Schema identifier (required):

```json theme={null}
{
  "name": "weather_response"
}
```

### `strict`

Enforce strict validation (default: `false`):

* `true`: Reject responses that don't match schema exactly
* `false`: Allow minor deviations

### `schema`

JSON Schema object defining the structure:

```json theme={null}
{
  "type": "object",
  "properties": {
    "field": {"type": "string"}
  },
  "required": ["field"],
  "additionalProperties": false
}
```

## JSON Schema Support

Supported JSON Schema features:

* Basic types: `string`, `number`, `integer`, `boolean`, `array`, `object`
* Constraints: `enum`, `minLength`, `maxLength`, `minimum`, `maximum`
* Nested objects and arrays
* `required` fields
* `additionalProperties` control

## Model Support

<Card title="View Structured Output Models" icon="book">
  For a complete list of models supporting structured outputs, visit [anannas.ai/models](https://anannas.ai/models) and filter by `json_mode` capability.
</Card>

Structured outputs are supported on:

* **OpenAI**: GPT-4, GPT-4 Turbo, GPT-5 Mini, GPT-5, o1, o3
* **Anthropic**: Claude 3 Opus, Claude 3 Sonnet, Claude Sonnet 4.5
* **Other providers**: Check `/v1/models` for `json_mode` capability

Query available models:

```bash theme={null}
curl https://api.anannas.ai/v1/models \
  -H "Authorization: Bearer $ANANNAS_API_KEY" | \
  jq '.data[] | select(.capabilities[] | contains("json_mode"))'
```

## Best Practices

1. **Property descriptions**: Guide the model with clear descriptions
2. **Use strict mode**: Set `strict: true` for production
3. **Keep schemas simple**: Simpler schemas yield more reliable results
4. **Required fields**: Explicitly mark required fields
5. **Type constraints**: Use `enum` for limited options

## Example: Type-Safe Response

```python theme={null}
from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.anannas.ai/v1",
    api_key="<ANANNAS_API_KEY>"
)

response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=[
        {"role": "user", "content": "Extract user info from: John Doe, age 30, email john@example.com"}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "user_info",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"},
                    "email": {"type": "string"}
                },
                "required": ["name", "age", "email"],
                "additionalProperties": False
            }
        }
    }
)

content = response.choices[0].message.content
user_data = json.loads(content)
# Type-safe: user_data["name"], user_data["age"], user_data["email"]
```

## Streaming with Structured Outputs

Structured outputs work with streaming:

```json theme={null}
{
  "stream": true,
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "weather",
      "strict": true,
      "schema": {...}
    }
  }
}
```

Accumulate content deltas and parse the complete JSON when the stream completes.

## Error Handling

Common errors:

1. **Unsupported model**: Error with `unsupported_parameter` type
2. **Invalid schema**: Request rejected before completion
3. **Schema violation**: Response fails validation in strict mode

```python theme={null}
try:
    response = client.chat.completions.create(...)
    content = json.loads(response.choices[0].message.content)
except json.JSONDecodeError:
    print("Failed to parse structured output")
except Exception as e:
    if "unsupported_parameter" in str(e):
        print("Model does not support structured outputs")
```

## Comparison: JSON Object vs JSON Schema

### JSON Object Mode

Simple JSON output without validation:

```json theme={null}
{
  "response_format": {
    "type": "json_object"
  }
}
```

### JSON Schema Mode

Strict validation with schema:

```json theme={null}
{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "response",
      "strict": true,
      "schema": {...}
    }
  }
}
```

## See Also

* [API Overview](/API/Overview) - Request format
* [Parameters](/API/Parameters) - `response_format` parameter


# Tool Calling
Source: https://docs.anannas.ai/Features/tool-calling

Function calling and tool execution with Anannas

## Overview

Tool calling (also called function calling) enables models to request execution of external functions. The model proposes tool calls; your application executes them and returns results to continue the conversation.

## How It Works

1. **Model proposes tool call** - Model identifies when a tool should be called
2. **Client executes tool** - Your application runs the function locally
3. **Return results** - Send tool output back in the conversation
4. **Model processes** - Model uses results to generate final response

## Request Format

Include `tools` array in your request:

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "messages": [
    {"role": "user", "content": "What's the weather in San Francisco?"}
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather in a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}
```

## Tool Definition Schema

```typescript theme={null}
type Tool = {
  type: "function";
  function: {
    name: string;
    description: string;
    parameters: {
      type: "object";
      properties: Record<string, JSONSchemaProperty>;
      required?: string[];
      additionalProperties?: boolean;
    };
  };
};
```

## Tool Choice

Control when tools are called:

* `"auto"`: Model decides (default)
* `"none"`: No tools called
* `"required"`: Model must call a tool
* `{type: "function", function: {name: string}}`: Force specific function

## Response Format

When a tool is called, the response includes `tool_calls`:

```json theme={null}
{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\": \"San Francisco\"}"
        }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}
```

## Executing Tools

Parse `arguments` (JSON string) and execute:

```python theme={null}
import json

tool_call = response["choices"][0]["message"]["tool_calls"][0]
function_name = tool_call["function"]["name"]
arguments = json.loads(tool_call["function"]["arguments"])

# Execute function
if function_name == "get_weather":
    result = get_weather(arguments["location"])
```

## Returning Tool Results

Send results as a `tool` message:

```json theme={null}
{
  "model": "openai/gpt-5-mini",
  "messages": [
    {"role": "user", "content": "What's the weather in NYC?"},
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\": \"New York City\"}"
        }
      }]
    },
    {
      "role": "tool",
      "content": "72°F, sunny",
      "tool_call_id": "call_123"
    }
  ],
  "tools": [...]
}
```

## Parallel Tool Calls

Enable multiple tool calls in one response:

```json theme={null}
{
  "parallel_tool_calls": true
}
```

Default: `true`. When enabled, model can call multiple tools simultaneously.

## Multi-Turn Conversations

Maintain conversation history including all tool calls and results:

```python theme={null}
messages = [
    {"role": "user", "content": "What's 25 * 17?"},
    {
        "role": "assistant",
        "content": None,
        "tool_calls": [{
            "id": "call_1",
            "type": "function",
            "function": {
                "name": "calculator",
                "arguments": "{\"operation\": \"multiply\", \"a\": 25, \"b\": 17}"
            }
        }]
    },
    {
        "role": "tool",
        "content": "425",
        "tool_call_id": "call_1"
    }
]

# Continue conversation
response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=messages,
    tools=[...]
)
```

## Supported Models

<Card title="View Tool-Calling Models" icon="book">
  For a complete list of models that support tool calling, visit [anannas.ai/models](https://anannas.ai/models) and filter by capability.
</Card>

Tool calling is supported on:

* **OpenAI**: GPT-4, GPT-3.5 Turbo, GPT-5 Mini, o1, o3
* **Anthropic**: Claude 3 Opus, Claude 3 Sonnet, Claude 3 Haiku, Claude Sonnet 4.5
* **Other providers**: Check `/v1/models` for `tool_calling` capability

## Best Practices

1. **Clear descriptions**: Write detailed function descriptions
2. **Required fields**: Specify `required` array for critical parameters
3. **Type constraints**: Use `enum` for limited options
4. **Error handling**: Handle invalid tool calls gracefully
5. **Conversation history**: Include all tool calls and results in subsequent requests

## Example: Complete Tool Calling Flow

```python theme={null}
from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.anannas.ai/v1",
    api_key="<ANANNAS_API_KEY>"
)

def get_weather(location: str, unit: str = "fahrenheit") -> str:
    # Simulate weather API
    return f"72°F, sunny in {location}"

messages = [
    {"role": "user", "content": "What's the weather in San Francisco?"}
]

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

# First request
response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=messages,
    tools=tools
)

message = response.choices[0].message
messages.append(message)

# Execute tool
if message.tool_calls:
    for tool_call in message.tool_calls:
        args = json.loads(tool_call.function.arguments)
        result = get_weather(args["location"])
        
        messages.append({
            "role": "tool",
            "content": result,
            "tool_call_id": tool_call.id
        })

# Get final response
response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=messages,
    tools=tools
)

print(response.choices[0].message.content)
```

## See Also

* [API Overview](/API/Overview) - Complete request/response schemas
* [Parameters](/API/Parameters) - Tool calling parameters


# Principles
Source: https://docs.anannas.ai/Principles

Core principles and values of Anannas

Anannas is Bhindi's *in-house* AI gateway that simplifies access to advanced models.  We believe the future of AI is **multi-modal, structured, and developer-first**.

### Why Anannas?

**Unified API.** A single endpoint powers all features - [structured outputs](/Features/structured-outputs), [multimodal inputs](/Features/Multimodality/Overview), [tool calling](/Features/tool-calling), and more. No code rewrites when you switch models.

**Model Routing.** Anannas intelligently routes requests across models for optimal performance, reliability, and cost efficiency - so your apps scale without friction.

**Structured Outputs.** Go beyond text: Anannas natively supports schema-driven responses, making it easy to plug results directly into your apps and workflows.

**Multimodality.** Handle text, images, and beyond in one place. Anannas is built to embrace the future of cross-modal AI.

**Developer Simplicity.** Bring your own API key or let Anannas manage access. Transparent billing and standardized responses make integration painless.

**Resilience.** Smart fallback and automatic rerouting ensure your applications stay online, even when individual models experience downtime.

**Future-Proof.** Built on Bhindi’s agentic AI stack, Anannas evolves alongside the ecosystem, so you always have access to the latest capabilities without code churn.

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Bring Your Own Keys (BYOK)
Source: https://docs.anannas.ai/UseCases/byok

Use your own provider API keys with Anannas for enhanced control over rate limits and billing.

With BYOK, you can route requests through your own provider accounts while still using Anannas as your unified gateway. This gives you direct visibility into provider-level usage and billing while letting Anannas handle the routing complexity.

Your keys are encrypted at rest and only used for requests to their respective providers. Anannas credits remain available as a fallback when your provider keys encounter issues; unless you choose otherwise.

Configure and manage your provider keys in [account settings](https://anannas.ai/dashboard/byok).

<Note>
  You can add multiple provider keys and toggle them on/off anytime. When a key is disabled, it won't be used for routing.
</Note>

## Getting started

<Steps>
  <Step title="Open Integrations (BYOK)">
    Navigate to the Integrations (BYOK) section in your dashboard sidebar.
  </Step>

  <Step title="Choose a provider">
    Select from the available providers.
  </Step>

  <Step title="Add your key">
    Click "Add key" and fill in the popup with Name and Key.
  </Step>

  <Step title="Configure options">
    Toggle "Enabled" to activate routing; optionally enable "Always use this key" to disable fallback.
  </Step>

  <Step title="Save">
    Save your configuration.
  </Step>
</Steps>

<Tip>
  Need help getting started with Anannas? Check the [QuickStart](/quickstart) guide for client setup and examples.
</Tip>

## Supported providers

<Tabs>
  <Tab title="Anthropic">
    In the Add key popup, provide:

    * **Name** — identifier for your key
    * **Key** — your Anthropic API key
    * **Enabled** — activate routing via your account
    * **Always use this key** — disable fallback to Anannas credits

    You can obtain your API key from your Anthropic account. See [Anthropic's docs](https://docs.anthropic.com/claude/reference/getting-started-with-the-api) for more info.
  </Tab>

  <Tab title="xAI (Grok)">
    In the Add key popup, provide:

    * **Name** — identifier for your key
    * **Key** — your xAI API key
    * **Enabled** — activate routing via your account
    * **Always use this key** — disable fallback to Anannas credits

    You can obtain your API key from your xAI account. Check out [xAI's docs](https://docs.x.ai/) for more info.
  </Tab>

  <Tab title="Nebius">
    In the Add key popup, provide:

    * **Name** — identifier for your key
    * **Key** — your Nebius API key
    * **Enabled** — activate routing via your account
    * **Always use this key** — disable fallback to Anannas credits

    You can obtain your API key from your Nebius account. See [Nebius docs](https://nebius.com/docs/) for more info.
  </Tab>
</Tabs>

<Info>
  We're actively expanding provider support. Want your provider listed? Reach out — we're happy to integrate.
</Info>

## Key priority and fallback

Anannas prioritizes your provider key when present and Enabled. If your key hits a rate limit or errors, we fall back to Anannas credits so your calls keep flowing.

Turn on "Always use this key" to disable fallback. With that enabled, we'll only use your key; you may see rate limit errors if your provider rejects the request.

## Pricing

<Card title="Check BYOK Pricing" icon="book">
  For current BYOK pricing and provider/model rates, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

<Warning>
  BYOK usage is billed at 5% of the underlying provider/model price, waived for the first 1M BYOK requests per month.

  Your provider still bills usage directly when your key is used; the 5% BYOK fee is separate and applies on the Anannas side.
</Warning>

## Using BYOK in clients

No code changes are required in your client — routing uses your configured keys automatically.

<CodeGroup>
  ```python python theme={null}
  from openai import OpenAI

  client = OpenAI(
    base_url="https://api.anannas.ai/v1",
    api_key="<ANANNAS_API_KEY>",
  )

  resp = client.chat.completions.create(
    model="anthropic/claude-3.7-sonnet",
    messages=[{"role": "user", "content": "Hello from Anannas!"}],
  )
  print(resp.choices[0].message.content)
  ```

  ```typescript typescript theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.anannas.ai/v1",
    apiKey: "<ANANNAS_API_KEY>",
  });

  const resp = await client.chat.completions.create({
    model: "openai/gpt-5-mini",
    messages: [{ role: "user", content: "Hello from Anannas!" }],
  });
  console.log(resp.choices[0].message.content);
  ```
</CodeGroup>

### When to enable "Always use this key"

Enable this option if you need strict guarantees that all requests go through your provider account (for auditing, compliance, or budget tracking). If disabled, Anannas falls back to platform credits when your key hits errors or limits; keeping your app running smoothly.

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Reasoning Tokens
Source: https://docs.anannas.ai/UseCases/reasoning



For models that support it, the Anannas API can return **Reasoning Tokens**, also known as thinking tokens. Anannas normalizes the different ways of customizing the amount of reasoning tokens that the model will use, providing a unified interface across different providers.

Reasoning tokens provide a transparent look into the reasoning steps taken by a model. Reasoning tokens are considered output tokens and charged accordingly.

Reasoning tokens are included in the response by default if the model decides to output them. Reasoning tokens will appear in the `reasoning` field of each message, unless you decide to exclude them.

<Note>
  **Some reasoning models do not return their reasoning tokens**

  While most models and providers make reasoning tokens available in the response, some (like the OpenAI o-series and Gemini Flash Thinking) do not.
</Note>

### **Controlling Reasoning Tokens**

You can control reasoning tokens in your requests using the `reasoning` parameter:

```json theme={null}
{
  "model": "your-model",
  "messages": [],
  "reasoning": {
    // One of the following (not both):
    "effort": "high", // Can be "high", "medium", or "low" (OpenAI-style)
    "max_tokens": 2000, // Specific token limit (Anthropic-style)

    // Optional: Default is false. All models support this.
    "exclude": false, // Set to true to exclude reasoning tokens from response

    // Or enable reasoning with the default parameters:
    "enabled": true // Default: inferred from `effort` or `max_tokens`
  }
}
```

The `reasoning` config object consolidates settings for controlling reasoning strength across different models. See the Note for each option below to see which models are supported and how other models will behave.

### **Max Tokens for Reasoning**

<Card title="Check Reasoning Support" icon="book">
  For models that support `max_tokens` reasoning configuration, visit [anannas.ai/models](https://anannas.ai/models) to see model-specific support.
</Card>

<Note>
  **Currently supported by:**

  * Gemini thinking models
  * Anthropic reasoning models (by using the `reasoning.max_tokens` parameter)
  * Some Alibaba Qwen thinking models (mapped to `thinking_budget`)

  For Alibaba, support varies by model — please check the individual model descriptions to confirm whether `reasoning.max_tokens` (via `thinking_budget`) is available.
</Note>

For models that support reasoning token allocation, you can control it like this:

* `"max_tokens": 2000` - Directly specifies the maximum number of tokens to use for reasoning

For models that only support `reasoning.effort` (see below), the `max_tokens` value will be used to determine the effort level.

### **Reasoning Effort Level**

<Card title="Check Reasoning Model Support" icon="book">
  For models that support reasoning and their specific capabilities, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

<Note>
  **Supported models**

  Currently supported by OpenAI reasoning models (o1 series, o3 series, GPT-5 series) and Grok models
</Note>

* `"effort": "high"` - Allocates a large portion of tokens for reasoning (approximately 80% of max\_tokens)
* `"effort": "medium"` - Allocates a moderate portion of tokens (approximately 50% of max\_tokens)
* `"effort": "low"` - Allocates a smaller portion of tokens (approximately 20% of max\_tokens)

For models that only support `reasoning.max_tokens`, the effort level will be set based on the percentages above.

### **Excluding Reasoning Tokens**

If you want the model to use reasoning internally but not include it in the response:

* `"exclude": true` - The model will still use reasoning, but it won’t be returned in the response

Reasoning tokens will appear in the `reasoning` field of each message.

### **Enable Reasoning with Default Config**

To enable reasoning with the default parameters:

* `"enabled": true` - Enables reasoning at the “medium” effort level with no exclusions.

We recommend using the new unified `reasoning` parameter for better control and future compatibility.

### **Examples**

### **Basic Usage with Reasoning Tokens**

<CodeGroup>
  ```python Python theme={null}
  import requests
  import json

  url = "https://api.anannas.ai/v1/chat/completions"
  headers = {
      "Authorization": f"Bearer <ANANNAS_API_KEY>",
      "Content-Type": "application/json"
  }
  payload = {
      "model": "openai/o3-mini",
      "messages": [
          {"role": "user", "content": "How would you build the world's tallest skyscraper?"}
      ],
      "reasoning": {
          "effort": "high"  # Use high reasoning effort
      }
  }

  response = requests.post(url, headers=headers, data=json.dumps(payload))
  print(response.json()['choices'][0]['message']['reasoning'])
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from 'openai';

  const openai = new OpenAI({
    baseURL: 'https://api.anannas.ai/v1',
    apiKey: '<ANANNAS_API_KEY>',
  });

  async function getResponseWithReasoning() {
    const response = await openai.chat.completions.create({
      model: 'openai/o3-mini',
      messages: [
        {
          role: 'user',
          content: "How would you build the world's tallest skyscraper?",
        },
      ],
      reasoning: {
        effort: 'high', // Use high reasoning effort
      },
    });

    console.log('REASONING:', response.choices[0].message.reasoning);
    console.log('CONTENT:', response.choices[0].message.content);
  }

  getResponseWithReasoning();
  ```
</CodeGroup>

### **Using Max Tokens for Reasoning**

For models that support direct token allocation (like Anthropic models), you can specify the exact number of tokens to use for reasoning:

<CodeGroup>
  ```python Python theme={null}
  import requests
  import json

  url = "https://api.anannas.ai/v1/chat/completions"
  headers = {
      "Authorization": f"Bearer <ANANNAS_API_KEY>",
      "Content-Type": "application/json"
  }
  payload = {
      "model": "anthropic/claude-3.7-sonnet",
      "messages": [
          {"role": "user", "content": "What's the most efficient algorithm for sorting a large dataset?"}
      ],
      "reasoning": {
          "max_tokens": 2000  # Allocate 2000 tokens (or approximate effort) for reasoning
      }
  }

  response = requests.post(url, headers=headers, data=json.dumps(payload))
  print(response.json()['choices'][0]['message']['reasoning'])
  print(response.json()['choices'][0]['message']['content'])
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from 'openai';

  const openai = new OpenAI({
    baseURL: 'https://api.anannas.ai/v1',
    apiKey: '<ANANNAS_API_KEY>',
  });

  async function getResponseWithReasoning() {
    const response = await openai.chat.completions.create({
      model: 'anthropic/claude-3.7-sonnet',
      messages: [
        {
          role: 'user',
          content: "How would you build the world's tallest skyscraper?",
        },
      ],
      reasoning: {
        max_tokens: 2000, // Allocate 2000 tokens (or approximate effort) for reasoning
      },
    });

    console.log('REASONING:', response.choices[0].message.reasoning);
    console.log('CONTENT:', response.choices[0].message.content);
  }

  getResponseWithReasoning();
  ```
</CodeGroup>

### **Excluding Reasoning Tokens from Response**

If you want the model to use reasoning internally but not include it in the response:

<CodeGroup>
  ```python Python theme={null}
  import requests
  import json

  url = "https://api.anannas.ai/v1/chat/completions"
  headers = {
      "Authorization": f"Bearer <ANANNAS_API_KEY>",
      "Content-Type": "application/json"
  }
  payload = {
      "model": "deepseek/deepseek-r1",
      "messages": [
          {"role": "user", "content": "Explain quantum computing in simple terms."}
      ],
      "reasoning": {
          "effort": "high",
          "exclude": true  # Use reasoning but don't include it in the response
      }
  }

  response = requests.post(url, headers=headers, data=json.dumps(payload))
  # No reasoning field in the response
  print(response.json()['choices'][0]['message']['content'])
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from 'openai';

  const openai = new OpenAI({
    baseURL: 'https://api.anannas.ai/v1',
    apiKey: '<ANANNAS_API_KEY>',
  });

  async function getResponseWithReasoning() {
    const response = await openai.chat.completions.create({
      model: 'deepseek/deepseek-r1',
      messages: [
        {
          role: 'user',
          content: "How would you build the world's tallest skyscraper?",
        },
      ],
      reasoning: {
        effort: 'high',
        exclude: true, // Use reasoning but don't include it in the response
      },
    });

    console.log('REASONING:', response.choices[0].message.reasoning);
    console.log('CONTENT:', response.choices[0].message.content);
  }

  getResponseWithReasoning();
  ```
</CodeGroup>

### **Advanced Usage: Reasoning Chain-of-Thought**

This example shows how to use reasoning tokens in a more complex workflow. It injects one model’s reasoning into another model to improve its response quality:

<CodeGroup>
  ```python Python theme={null}
  import requests
  import json

  question = "Which is bigger: 9.11 or 9.9?"

  url = "https://api.anannas.ai/v1/chat/completions"
  headers = {
      "Authorization": f"Bearer <ANANNAS_API_KEY>",
      "Content-Type": "application/json"
  }

  def do_req(model, content, reasoning_config=None):
      payload = {
          "model": model,
          "messages": [
              {"role": "user", "content": content}
          ],
          "stop": "</think>"
      }

      return requests.post(url, headers=headers, data=json.dumps(payload))

  # Get reasoning from a capable model
  content = f"{question} Please think this through, but don't output an answer"
  reasoning_response = do_req("deepseek/deepseek-r1", content)
  reasoning = reasoning_response.json()['choices'][0]['message']['reasoning']

  # Let's test! Here's the naive response:
  simple_response = do_req("openai/gpt-5-mini", question)
  print(simple_response.json()['choices'][0]['message']['content'])

  # Here's the response with the reasoning token injected:
  content = f"{question}. Here is some context to help you: {reasoning}"
  smart_response = do_req("openai/gpt-5-mini", content)
  print(smart_response.json()['choices'][0]['message']['content'])
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from 'openai';

  const openai = new OpenAI({
    baseURL: 'https://api.anannas.ai/v1',
    apiKey,
  });

  async function doReq(model, content, reasoningConfig) {
    const payload = {
      model,
      messages: [{ role: 'user', content }],
      stop: '</think>',
      ...reasoningConfig,
    };

    return openai.chat.completions.create(payload);
  }

  async function getResponseWithReasoning() {
    const question = 'Which is bigger: 9.11 or 9.9?';
    const reasoningResponse = await doReq(
      'deepseek/deepseek-r1',
      `${question} Please think this through, but don't output an answer`,
    );
    const reasoning = reasoningResponse.choices[0].message.reasoning;

    // Let's test! Here's the naive response:
    const simpleResponse = await doReq('openai/gpt-5-mini', question);
    console.log(simpleResponse.choices[0].message.content);

    // Here's the response with the reasoning token injected:
    const content = `${question}. Here is some context to help you: ${reasoning}`;
    const smartResponse = await doReq('openai/gpt-5-mini', content);
    console.log(smartResponse.choices[0].message.content);
  }

  getResponseWithReasoning();
  ```
</CodeGroup>

## **Provider-Specific Reasoning Implementation**

### **Anthropic Models with Reasoning Tokens**

The latest Claude models, such as [**<u>anthropic/claude-3.7-sonnet</u>**](https://anannas.ai/models/anthropic/claude-3.7-sonnet), support working with and returning reasoning tokens.

You can enable reasoning on Anthropic models **only** using the unified `reasoning`parameter with either `effort` or `max_tokens`.

**Note:** The `:thinking` variant is no longer supported for Anthropic models. Use the `reasoning` parameter instead.

#### **Reasoning Max Tokens for Anthropic Models**

<Card title="Check Reasoning Limits" icon="book">
  For model-specific reasoning token limits and allocation rules, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

When using Anthropic models with reasoning:

* When using the `reasoning.max_tokens` parameter, that value is used directly with a minimum of 1024 tokens.
* When using the `reasoning.effort` parameter, the budget\_tokens are calculated based on the `max_tokens` value.

The reasoning token allocation is capped at 32,000 tokens maximum and 1024 tokens minimum. The formula for calculating the budget\_tokens is: `budget_tokens = max(min(max_tokens * {effort_ratio}, 32000), 1024)`

effort\_ratio is 0.8 for high effort, 0.5 for medium effort, and 0.2 for low effort.

**Important**:

* When using reasoning **without tools**, `max_tokens` must be strictly higher than the reasoning budget to ensure there are tokens available for the final response after thinking.
* When using reasoning **with tools** (interleaved thinking), the `budget_tokens` can exceed `max_tokens` as it represents the total budget across all thinking blocks within one assistant turn.

**Restrictions when reasoning is enabled:**

* `tool_choice` can only be `"auto"` or `"none"` (not `"required"` or specific tool)
* `top_k` parameter is not allowed
* Pre-filled assistant messages (assistant messages with content) are not allowed

<Note>
  **Token Usage and Billing**

  Please note that reasoning tokens are counted as output tokens for billing purposes. Using reasoning tokens will increase your token usage but can significantly improve the quality of model responses.
</Note>

#### **Interleaved Thinking with Tool Use**

<Card title="Model Support" icon="book">
  Interleaved thinking is automatically enabled for Claude 4 models (Sonnet 4.5, Opus 4.5, Haiku 4.5) when reasoning is enabled with tools. Visit [anannas.ai/models](https://anannas.ai/models) to see which models support this feature.
</Card>

Interleaved thinking allows the model to reason between tool calls, enabling more sophisticated reasoning after receiving tool results. This feature is automatically enabled when:

* Reasoning/thinking is enabled (`reasoning.max_tokens` or `reasoning.effort`)
* Tools are present in the request
* Tool choice is `"auto"` (or not specified, which defaults to auto)

**Key Benefits:**

* **Reasoning between tool calls**: The model can think about tool results before deciding what to do next
* **Chained tool calls**: Enables more nuanced decisions based on intermediate results
* **Flexible token allocation**: With interleaved thinking, `budget_tokens` can exceed `max_tokens` as it represents the total budget across all thinking blocks

**Example: Interleaved Thinking with Tools**

<CodeGroup>
  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.anannas.ai/v1",
      api_key="<ANANNAS_API_KEY>",
  )

  response = client.chat.completions.create(
      model="anthropic/claude-sonnet-4-5-20250929",
      messages=[
          {
              "role": "user",
              "content": "What's the weather in Paris? Also tell me the current time there."
          }
      ],
      tools=[
          {
              "type": "function",
              "function": {
                  "name": "get_weather",
                  "description": "Get weather for a location",
                  "parameters": {
                      "type": "object",
                      "properties": {
                          "location": {"type": "string"}
                      },
                      "required": ["location"]
                  }
              }
          },
          {
              "type": "function",
              "function": {
                  "name": "get_time",
                  "description": "Get current time for a location",
                  "parameters": {
                      "type": "object",
                      "properties": {
                          "location": {"type": "string"}
                      },
                      "required": ["location"]
                  }
              }
          }
      ],
      tool_choice="auto",  # Required for interleaved thinking
      reasoning={
          "max_tokens": 5000  # Can exceed max_tokens when interleaved thinking is enabled
      },
      max_tokens=2000
  )

  # The model will reason before each tool call and after receiving results
  print(response.choices[0].message.reasoning)
  print(response.choices[0].message.tool_calls)
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from 'openai';

  const client = new OpenAI({
    baseURL: 'https://api.anannas.ai/v1',
    apiKey: '<ANANNAS_API_KEY>',
  });

  const response = await client.chat.completions.create({
    model: 'anthropic/claude-sonnet-4-5-20250929',
    messages: [
      {
        role: 'user',
        content: "What's the weather in Paris? Also tell me the current time there.",
      },
    ],
    tools: [
      {
        type: 'function',
        function: {
          name: 'get_weather',
          description: 'Get weather for a location',
          parameters: {
            type: 'object',
            properties: {
              location: { type: 'string' },
            },
            required: ['location'],
          },
        },
      },
      {
        type: 'function',
        function: {
          name: 'get_time',
          description: 'Get current time for a location',
          parameters: {
            type: 'object',
            properties: {
              location: { type: 'string' },
            },
            required: ['location'],
          },
        },
      },
    ],
    tool_choice: 'auto', // Required for interleaved thinking
    reasoning: {
      max_tokens: 5000, // Can exceed max_tokens when interleaved thinking is enabled
    },
    max_tokens: 2000,
  });

  // The model will reason before each tool call and after receiving results
  console.log(response.choices[0].message.reasoning);
  console.log(response.choices[0].message.tool_calls);
  ```
</CodeGroup>

<Note>
  **Tool Choice Restrictions with Thinking**

  When reasoning is enabled, only `tool_choice: "auto"` or `tool_choice: "none"` are supported. Using `tool_choice: "required"` or forcing a specific tool will result in an error.
</Note>

### **Examples with Anthropic Models**

#### **Example 1: Streaming mode with reasoning tokens**

<CodeGroup>
  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.anannas.ai/v1",
      api_key="<ANANNAS_API_KEY>",
  )

  def chat_completion_with_reasoning(messages):
      response = client.chat.completions.create(
          model="anthropic/claude-3.7-sonnet",
          messages=messages,
          max_tokens=10000,
          reasoning={
              "max_tokens": 8000  # Directly specify reasoning token budget
          },
          stream=True
      )
      return response

  for chunk in chat_completion_with_reasoning([
      {"role": "user", "content": "What's bigger, 9.9 or 9.11?"}
  ]):
      if hasattr(chunk.choices[0].delta, 'reasoning') and chunk.choices[0].delta.reasoning:
          print(f"REASONING: {chunk.choices[0].delta.reasoning}")
      elif chunk.choices[0].delta.content:
          print(f"CONTENT: {chunk.choices[0].delta.content}")
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from 'openai';

  const openai = new OpenAI({
    baseURL: 'https://api.anannas.ai/v1',
    apiKey,
  });

  async function chatCompletionWithReasoning(messages) {
    const response = await openai.chat.completions.create({
      model: '{{MODEL}}',
      messages,
      maxTokens: 10000,
      reasoning: {
        maxTokens: 8000, // Directly specify reasoning token budget
      },
      stream: true,
    });

    return response;
  }

  (async () => {
    for await (const chunk of chatCompletionWithReasoning([
      { role: 'user', content: "What's bigger, 9.9 or 9.11?" },
    ])) {
      if (chunk.choices[0].delta.reasoning) {
        console.log(`REASONING: ${chunk.choices[0].delta.reasoning}`);
      } else if (chunk.choices[0].delta.content) {
        console.log(`CONTENT: ${chunk.choices[0].delta.content}`);
      }
    }
  })();
  ```
</CodeGroup>

### **Preserving Reasoning Blocks**

<Note>
  **Model Support**

  The reasoning\_details are currently returned by all OpenAI reasoning models (o1 series, o3 series, GPT-5 series) and all Anthropic reasoning models (Claude 3.7, Claude 4, and Claude 4.1 series).
</Note>

The reasoning\_details functionality works identically across all supported reasoning models. You can easily switch between OpenAI reasoning models (like `openai/gpt-5-mini`) and Anthropic reasoning models (like `anthropic/claude-sonnet-4`) without changing your code structure.

If you want to pass reasoning back in context, you must pass reasoning blocks back to the API. This is useful for maintaining the model’s reasoning flow and conversation integrity.

Preserving reasoning blocks is useful specifically for tool calling. When models like Claude invoke tools, it is pausing its construction of a response to await external information. When tool results are returned, the model will continue building that existing response. This necessitates preserving reasoning blocks during tool use, for a couple of reasons:

**Reasoning continuity**: The reasoning blocks capture the model’s step-by-step reasoning that led to tool requests. When you post tool results, including the original reasoning ensures the model can continue its reasoning from where it left off.

**Context maintenance**: While tool results appear as user messages in the API structure, they’re part of a continuous reasoning flow. Preserving reasoning blocks maintains this conceptual flow across multiple API calls.

<Note>
  **Important for Reasoning Models**

  When providing reasoning\_details blocks, the entire sequence of consecutive reasoning blocks must match the outputs generated by the model during the original request; you cannot rearrange or modify the sequence of these blocks.
</Note>

## **Responses API Shape**

When reasoning models generate responses, the reasoning information is structured in a standardized format through the `reasoning_details` array. This section documents the API response structure for reasoning details in both streaming and non-streaming responses, based on the schema definitions in the `llm-interfaces` package.

### **reasoning\_details Array Structure**

The `reasoning_details` field contains an array of reasoning detail objects. Each object in the array represents a specific piece of reasoning information and follows one of three possible types. The location of this array differs between streaming and non-streaming responses:

* **Non-streaming responses**: `reasoning_details` appears in `choices[].message.reasoning_details`
* **Streaming responses**: `reasoning_details` appears in `choices[].delta.reasoning_details` for each chunk

#### **Common Fields**

All reasoning detail objects share these common fields:

* `id` (string | null): Unique identifier for the reasoning detail
* `format` (string): The format of the reasoning detail, with possible values:
  * `"unknown"` - Format is not specified
  * `"openai-responses-v1"` - OpenAI responses format version 1
  * `"anthropic-claude-v1"` - Anthropic Claude format version 1 (default)
* `index` (number, optional): Sequential index of the reasoning detail

#### **Reasoning Detail Types**

**1. Summary Type (**`reasoning.summary`**)**

Contains a high-level summary of the reasoning process:

```json theme={null}
{
  "type": "reasoning.summary",
  "summary": "The model analyzed the problem by first identifying key constraints, then evaluating possible solutions...",
  "id": "reasoning-summary-1",
  "format": "anthropic-claude-v1",
  "index": 0
}
```

**2. Encrypted Type (**`reasoning.encrypted`**)**

Contains encrypted reasoning data that may be redacted or protected:

```json theme={null}
{  
	"type": "reasoning.encrypted",  
	"data": "eyJlbmNyeXB0ZWQiOiJ0cnVlIiwiY29udGVudCI6IltSRURBQ1RFRF0ifQ==",  
	"id": "reasoning-encrypted-1",  
	"format": "anthropic-claude-v1",  
	"index": 1
}
```

**3. Text Type (**`reasoning.text`**)**

Contains raw text reasoning with optional signature verification:

```json theme={null}
{  
	"type": "reasoning.text",  
	"text": "Let me think through this step by step:\n1. First, I need to understand the user's question...",  
	"signature": "sha256:abc123def456...",  
	"id": "reasoning-text-1",  
	"format": "anthropic-claude-v1",  
	"index": 2
}
```

### **Response Examples**

#### **Non-Streaming Response**

In non-streaming responses, `reasoning_details` appears in the message:

<CodeGroup>
  ```json filename theme={null}
  // add code here{
    "choices": [
      {
        "message": {
          "role": "assistant",
          "content": "Based on my analysis, I recommend the following approach...",
          "reasoning_details": [
            {
              "type": "reasoning.summary",
              "summary": "Analyzed the problem by breaking it into components",
              "id": "reasoning-summary-1",
              "format": "anthropic-claude-v1",
              "index": 0
            },
            {
              "type": "reasoning.text",
              "text": "Let me work through this systematically:\n1. First consideration...\n2. Second consideration...",
              "signature": null,
              "id": "reasoning-text-1",
              "format": "anthropic-claude-v1",
              "index": 1
            }
          ]
        }
      }
    ]
  }
  ```
</CodeGroup>

#### **Streaming Response**

In streaming responses, `reasoning_details` appears in delta chunks as the reasoning is generated:

<CodeGroup>
  ```json filename theme={null}
  {
    "choices": [
      {
        "delta": {
          "reasoning_details": [
            {
              "type": "reasoning.text",
              "text": "Let me think about this step by step...",
              "signature": null,
              "id": "reasoning-text-1",
              "format": "anthropic-claude-v1",
              "index": 0
            }
          ]
        }
      }
    ]
  }
  ```
</CodeGroup>

**Streaming Behavior Notes:**

* Each reasoning detail chunk is sent as it becomes available
* The `reasoning_details` array in each chunk may contain one or more reasoning objects
* For encrypted reasoning, the content may appear as `[REDACTED]` in streaming responses
* The complete reasoning sequence is built by concatenating all chunks in order

### **Example: Preserving Reasoning Blocks with Anannas AI and Claude**

<CodeGroup>
  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.anannas.ai/v1",
      api_key="<ANANNAS_API_KEY>",
  )

  # First API call with tools
  # Note: You can use 'openai/gpt-5-mini' instead of 'anthropic/claude-sonnet-4' - they're completely interchangeable
  response = client.chat.completions.create(
      model="anthropic/claude-sonnet-4",
      messages=[
          {"role": "user", "content": "What's the weather like in Boston? Then recommend what to wear."}
      ],
      tools=[{
          "type": "function",
          "function": {
              "name": "get_weather",
              "description": "Get current weather",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "location": {"type": "string"}
                  },
                  "required": ["location"]
              }
          }
      }],
      reasoning={"max_tokens": 2000}
  )

  # Extract the assistant message with reasoning_details
  message = response.choices[0].message

  # Preserve the complete reasoning_details when passing back
  messages = [
      {"role": "user", "content": "What's the weather like in Boston? Then recommend what to wear."},
      {
          "role": "assistant",
          "content": message.content,
          "tool_calls": message.tool_calls,
          "reasoning_details": message.reasoning_details  # Pass back unmodified
      },
      {
          "role": "tool",
          "tool_call_id": message.tool_calls[0].id,
          "content": '{"temperature": 45, "condition": "rainy", "humidity": 85}'
      }
  ]

  # Second API call - Claude continues reasoning from where it left off
  response2 = client.chat.completions.create(
      model="anthropic/claude-sonnet-4",
      messages=messages,  # Includes preserved thinking blocks
      tools=tools
  )
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from 'openai';

  const client = new OpenAI({
    baseURL: 'https://api.anannas.ai/v1',
    apiKey: '<ANANNAS_API_KEY>',
  });

  // First API call with tools
  // Note: You can use 'openai/gpt-5-mini' instead of 'anthropic/claude-sonnet-4' - they're completely interchangeable
  const response = await client.chat.completions.create({
    model: 'anthropic/claude-sonnet-4',
    messages: [
      {
        role: 'user',
        content:
          "What's the weather like in Boston? Then recommend what to wear.",
      },
    ],
    tools: [
      {
        type: 'function',
        function: {
          name: 'get_weather',
          description: 'Get current weather',
          parameters: {
            type: 'object',
            properties: {
              location: { type: 'string' },
            },
            required: ['location'],
          },
        },
      },
    ],
    reasoning: { max_tokens: 2000 },
  });

  // Extract the assistant message with reasoning_details
  const message = response.choices[0].message;

  // Preserve the complete reasoning_details when passing back
  const messages = [
    {
      role: 'user',
      content: "What's the weather like in Boston? Then recommend what to wear.",
    },
    {
      role: 'assistant',
      content: message.content,
      tool_calls: message.tool_calls,
      reasoning_details: message.reasoning_details, // Pass back unmodified
    },
    {
      role: 'tool',
      tool_call_id: message.tool_calls[0].id,
      content: JSON.stringify({
        temperature: 45,
        condition: 'rainy',
        humidity: 85,
      }),
    },
  ];

  // Second API call - Claude continues reasoning from where it left off
  const response2 = await client.chat.completions.create({
    model: 'anthropic/claude-sonnet-4',
    messages, // Includes preserved thinking blocks
    tools,
  });
  ```
</CodeGroup>

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Usage Accounting
Source: https://docs.anannas.ai/UseCases/usage-acc



The Anannas API provides *built-in* **Usage Accounting** that allows you to track AI model usage without making additional API calls. This feature provides detailed information about token counts, costs, and caching status directly in your API responses.

### Usage Information

When enabled, the API will return detailed usage information including:

1. Prompt and completion token counts using the model’s native tokenizer
2. Cost in credits
3. Reasoning token counts (if applicable)
4. Cached token counts (if available)

This information is included in the last SSE message for streaming responses, or in the complete response for non-streaming requests.

### Enabling Usage Accounting

You can enable usage accounting in your requests by including the `usage` parameter:

```bash theme={null}
{
  "model": "your-model",
  "messages": [],
  "usage": {
    "include": true
  }
}
```

### Response Format

When usage accounting is enabled, the response will include a `usage` object with detailed token information:

```bash theme={null}
{
  "object": "chat.completion.chunk",
  "usage": {
    "completion_tokens": 2,
    "completion_tokens_details": {
      "reasoning_tokens": 0
    },
    "cost": 0.95,
    "cost_details": {
      "upstream_inference_cost": 19
    },
    "prompt_tokens": 194,
    "prompt_tokens_details": {
      "cached_tokens": 0,
      "audio_tokens": 0
    },
    "total_tokens": 196
  }
}
```

`cached_tokens` is the number of tokens that were *read* from the cache. At this point in time, we do not support retrieving the number of tokens that were *written* to the cache.

### Cost Breakdown

The usage response includes detailed cost information:\
`cost`: The total amount charged to your account

<Note>
  Performance Impact

  Enabling usage accounting will add a few hundred milliseconds to the last response as the API calculates token counts and costs. This only affects the final message and does not impact overall streaming performance.
</Note>

### Benefits

1. **Efficiency**: Get usage information without making separate API calls
2. **Accuracy**: Token counts are calculated using the model’s native tokenizer
3. **Transparency**: Track costs and cached token usage in real-time
4. **Detailed Breakdown**: Separate counts for prompt, completion, reasoning, and cached tokens

### Best Practices

1. Enable usage tracking when you need to monitor token consumption or costs
2. Account for the slight delay in the final response when usage accounting is enabled
3. Consider implementing usage tracking in development to optimize token usage before production
4. Use the cached token information to optimize your application’s performance

<div>
  <span>Was this page helpful?</span>

  <button>
    <span>👍</span>
    Yes
  </button>

  <button>
    <span>👎</span>
    No
  </button>
</div>


# Models
Source: https://docs.anannas.ai/models

Available models and capabilities

## Overview

Anannas provides access to models from multiple providers through a unified API. Models are identified by `provider/model-name` format.

## Model Endpoint

Query available models:

```
GET https://api.anannas.ai/v1/models
```

### Response Format

```json theme={null}
{
  "data": [
    {
      "id": "openai/gpt-5-mini",
      "name": "GPT-5 Mini",
      "provider": "openai",
      "description": "Most capable GPT-5 Mini model",
      "context_length": 8192,
      "pricing": {
        "prompt": "0.00003",
        "completion": "0.00006",
        "request": "0"
      },
      "capabilities": [
        "text",
        "tool_calling",
        "json_mode"
      ]
    }
  ]
}
```

## Model Object Schema

| Field            | Type       | Description                                       |
| ---------------- | ---------- | ------------------------------------------------- |
| `id`             | `string`   | Model identifier (e.g., `openai/gpt-5-mini`)      |
| `name`           | `string`   | Human-readable name                               |
| `provider`       | `string`   | Provider identifier (`openai`, `anthropic`, etc.) |
| `description`    | `string`   | Model description                                 |
| `context_length` | `number`   | Maximum context window in tokens                  |
| `pricing`        | `object`   | Cost structure (see below)                        |
| `capabilities`   | `string[]` | Supported features                                |

## Pricing Object

```typescript theme={null}
type Pricing = {
  prompt: string;      // Cost per input token (USD)
  completion: string; // Cost per output token (USD)
  request?: string;   // Fixed cost per request (USD)
};
```

Prices are returned as strings to avoid floating-point precision issues.

## Model Format

Models use the format `provider/model-name`:

* `openai/gpt-5-mini`
* `openai/gpt-3.5-turbo`
* `anthropic/claude-3-sonnet`
* `anthropic/claude-3-opus`
* `x-ai/grok-beta`
* `groq/llama-3-70b`

## Capabilities

<Card title="View Model Capabilities" icon="book">
  For complete capability listings by model, visit [anannas.ai/models](https://anannas.ai/models) to see which features each model supports.
</Card>

Models may support:

* `text`: Text generation
* `tool_calling`: Function/tool execution
* `json_mode`: Structured JSON output
* `streaming`: Server-Sent Events
* `multimodal`: Image/audio inputs
* `reasoning`: Extended reasoning (o1, o3, etc.)
* `audio_output`: Audio generation
* `image_output`: Image generation

## Provider-Specific Models

<Card title="View All Models" icon="book">
  For complete, up-to-date model listings, pricing, capabilities, and parameter support, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

### OpenAI

* `openai/gpt-5-mini`
* `openai/gpt-4-turbo`
* `openai/gpt-3.5-turbo`
* `openai/o1-preview`
* `openai/o1-mini`
* `openai/o3-mini`

### Anthropic

* `anthropic/claude-3-opus`
* `anthropic/claude-3-sonnet`
* `anthropic/claude-3-haiku`
* `anthropic/claude-sonnet-4-5`

### xAI (Grok)

* `x-ai/grok-beta`
* `x-ai/grok-2`

### Other Providers

Models from Groq, TogetherAI, DeepInfra, Fireworks, Nebius, and others are available. Query `/v1/models` for the complete list.

## Model Selection

### Default Model

If `model` is omitted, Anannas uses your account's default model. Set defaults in the dashboard.

### Model Routing

Use `provider` preferences to control model selection:

```json theme={null}
{
    "model": "openai/gpt-5-mini",
  "provider": {
    "order": ["openai", "anthropic"],
    "sort": "price"
  }
}
```

### Fallbacks

Specify fallback models:

```json theme={null}
{
    "model": "openai/gpt-5-mini",
  "fallbacks": [
    "anthropic/claude-3-sonnet",
    "openai/gpt-3.5-turbo"
  ]
}
```

## Parameter Support

<Card title="Check Parameter Support" icon="book">
  For detailed parameter support by model, visit [anannas.ai/models](https://anannas.ai/models) to see which parameters each model supports.
</Card>

Not all models support all parameters. Common support:

* **Temperature**: Most models
* **Top P**: Most models
* **Max Tokens**: All models
* **Tool Calling**: OpenAI, Anthropic (Claude 3+)
* **JSON Mode**: OpenAI GPT-5 Mini, Claude 3+
* **Streaming**: All models
* **Reasoning**: o1, o3, Claude Sonnet 4.5

## Context Windows

<Card title="View Context Windows" icon="book">
  For accurate context window sizes for all models, check [anannas.ai/models](https://anannas.ai/models).
</Card>

Context window sizes vary by model:

* GPT-5 Mini: 128,000 tokens
* GPT-4 Turbo: 128,000 tokens
* Claude 3 Opus: 200,000 tokens
* Claude 3 Sonnet: 200,000 tokens
* o1: 128,000 tokens

Exceeding context limits returns an error.

## Rate Limits

Rate limits are tier-based and vary by model. Check your tier in the dashboard. Limits apply per API key.

## Pricing

<Card title="View Current Pricing" icon="book">
  For up-to-date pricing for all models, visit [anannas.ai/models](https://anannas.ai/models). Pricing is updated in real-time.
</Card>

Pricing is per-token and varies by model. Check the `/v1/models` endpoint for current rates. Prices are in USD.

Example pricing (approximate):

* GPT-5 Mini: $0.0015/1K input, $0.002/1K output
* GPT-3.5 Turbo: $0.0015/1K input, $0.002/1K output
* Claude 3 Sonnet: $0.003/1K input, $0.015/1K output

## Model Updates

Models are updated automatically. Model identifiers remain stable, but underlying model versions may change. Check model descriptions for version information.

## Deprecated Models

Deprecated models continue to work but may be removed in future versions. Migrate to supported alternatives.

## See Also

* [API Overview](/API/Overview) - Request format
* [Parameters](/API/Parameters) - Parameter support
* [Quickstart](/quickstart) - Getting started


# Quickstart
Source: https://docs.anannas.ai/quickstart

Get started with the Anannas API

## Overview

Anannas provides a unified OpenAI-compatible API for accessing multiple LLM providers. The Anannas API endpoint is `https://api.anannas.ai/v1` and uses standard HTTP POST requests with JSON payloads.

## Authentication

All requests require a Bearer token in the `Authorization` header. Get your API key from the [Anannas dashboard](https://anannas.ai/dashboard).

```bash theme={null}
Authorization: Bearer <ANANNAS_API_KEY>
```

## Endpoint

```
POST https://api.anannas.ai/v1/chat/completions
```

## Request Format

The Anannas API follows the OpenAI Chat Completions format. Required fields:

* `model` (string): Model identifier in format `provider/model-name` (e.g., `openai/gpt-5-mini`, `anthropic/claude-3-sonnet`)
* `messages` (array): Array of message objects with `role` and `content` fields

## Using the OpenAI SDK

The official OpenAI SDK works with Anannas by setting the `base_url` parameter:

<CodeGroup>
  ```python python theme={null}
  from openai import OpenAI

  client = OpenAI(
    base_url="https://api.anannas.ai/v1",
    api_key="<ANANNAS_API_KEY>",
  )

  response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=[
      { "role": "user", "content": "Explain quantum computing" }
    ]
  )

  print(response.choices[0].message.content)
  ```

  ```typescript typescript theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.anannas.ai/v1",
    apiKey: "<ANANNAS_API_KEY>",
  });

  const completion = await client.chat.completions.create({
    model: "anthropic/claude-3-sonnet",
    messages: [
      { role: "user", content: "Explain quantum computing" },
    ],
  });

  console.log(completion.choices[0].message.content);
  ```
</CodeGroup>

## Direct HTTP Requests

<CodeGroup>
  ```python python theme={null}
  import requests

  response = requests.post(
    "https://api.anannas.ai/v1/chat/completions",
    headers={
      "Authorization": "Bearer <ANANNAS_API_KEY>",
      "Content-Type": "application/json",
    },
    json={
      "model": "openai/gpt-5-mini",
      "messages": [
        { "role": "user", "content": "Explain quantum computing" }
      ]
    }
  )

  data = response.json()
  print(data["choices"][0]["message"]["content"])
  ```

  ```typescript typescript theme={null}
  const response = await fetch("https://api.anannas.ai/v1/chat/completions", {
    method: "POST",
    headers: {
      "Authorization": "Bearer <ANANNAS_API_KEY>",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "anthropic/claude-3-sonnet",
      messages: [
        { role: "user", content: "Explain quantum computing" },
      ],
    }),
  });

  const data = await response.json();
  console.log(data.choices[0].message.content);
  ```

  ```bash bash theme={null}
  curl https://api.anannas.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $ANANNAS_API_KEY" \
    -d '{
      "model": "openai/gpt-5-mini",
      "messages": [
        {"role": "user", "content": "Explain quantum computing"}
      ]
    }'
  ```
</CodeGroup>

## Response Format

Responses follow the OpenAI Chat Completions schema:

```json theme={null}
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "openai/gpt-5-mini",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Quantum computing is..."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 50,
    "total_tokens": 60
  }
}
```

## Streaming

Enable streaming by setting `stream: true`. The Anannas API returns Server-Sent Events (SSE) with incremental text deltas. See the [Streaming documentation](/API/Streaming) for implementation details.

## Model Format

<Card title="Browse All Models" icon="book">
  For a complete list of available models with pricing and capabilities, visit [anannas.ai/models](https://anannas.ai/models).
</Card>

Models are specified as `provider/model-name`:

* `openai/gpt-5-mini`
* `openai/gpt-3.5-turbo`
* `anthropic/claude-3-sonnet`
* `anthropic/claude-3-opus`

Query available models via `GET /v1/models` or see the [models documentation](/models).

## Error Handling

The Anannas API returns standard HTTP status codes:

* `200`: Success
* `400`: Invalid request (malformed JSON, missing required fields)
* `401`: Authentication failed (invalid or missing API key)
* `402`: Insufficient credits
* `429`: Rate limit exceeded
* `500`: Internal server error

Error responses follow this format:

```json theme={null}
{
  "error": {
    "message": "Invalid model: unknown-model",
    "type": "invalid_request_error"
  }
}
```

## Next Steps

* Read the [API Overview](/API/Overview) for complete request/response schemas
* Review [Parameters](/API/Parameters) for all available options
* Check [Authentication](/API/Authentication) for security best practices
* Explore [Streaming](/API/Streaming) for real-time responses