Batch API

Submit up to a thousand requests at once and collect the answers within 24 hours, at half the model's live price. For evaluations, backfills, classification runs and anything else that does not need an answer while someone waits.

API only. Batches are submitted and read through/api/v1/batches with an API key. There is no batch mode in the chat apps: a model whose id ends in :batch is the batch price tag for its base model, and the apps mark it API only rather than offering it in the picker.

How it differs from OpenAI's

Requests go inline in the create call, and results come back inline on the completed batch. There are no input or output files and no /filesendpoint, so the OpenAI SDK's batches.create(input_file_id=…) does not apply. A plain HTTP call does; examples below.
One model per batch, named once on the batch. Use the plain id (openai/gpt-6-astra) or the :batch id; the batch rate applies either way.
The completion window is 24 hours and cannot be shortened. A batch cannot be cancelled once accepted.

Credit

Batches spend purchased credit only: they are charged when they finish, up to a day after the daily allowance that was checked has reset. Before accepting a batch the gateway estimates its cost at the batch rate, summed over every request, and refuses with 402 insufficient_credits_for_batch if your purchased balance is below it. The actual charge is what the batch used, at the batch rate, and appears on your usage page against the :batch model id.

Create a batch

POST /api/v1/batches. Each request carries a custom_id that is unique within the batch and a body in the shape of the endpoint it targets: /v1/chat/completions (the default) or /v1/responses.

Bash

curl https://mume.ai/api/v1/batches \
  -H "Authorization: Bearer $MUME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-6-astra",
    "endpoint": "/v1/chat/completions",
    "requests": [
      { "custom_id": "review-1",
        "body": { "messages": [{"role": "user", "content": "Summarise: …"}],
                  "max_tokens": 200 } },
      { "custom_id": "review-2",
        "body": { "messages": [{"role": "user", "content": "Summarise: …"}],
                  "max_tokens": 200 } }
    ],
    "metadata": { "job": "nightly-reviews" }
  }'

Python

import requests

r = requests.post(
    "https://mume.ai/api/v1/batches",
    headers={"Authorization": f"Bearer {MUME_API_KEY}"},
    json={
        "model": "openai/gpt-6-astra",
        "requests": [
            {"custom_id": f"row-{i}",
             "body": {"messages": [{"role": "user", "content": text}],
                      "max_tokens": 200}}
            for i, text in enumerate(rows)
        ],
    },
)
batch = r.json()
print(batch["id"], batch["status"])   # batch-…, validating

Response

202 Accepted with the batch object. status moves through validating, in_progress, finalizing to completed; failed and expired are the other ends.

JSON

{
  "id": "batch-1789978582-jGhOYK4d",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "model": "openai/gpt-6-astra:batch",
  "status": "validating",
  "completion_window": "24h",
  "created_at": 1789978582,
  "request_counts": { "total": 2, "completed": 0, "failed": 0 },
  "metadata": { "job": "nightly-reviews" }
}

Poll and collect

GET /api/v1/batches/:id returns the batch; once status is completed it carries results inline, one entry per request, in no particular order. Match them up by custom_id. Each entry has either a response (with status_code and the body that endpoint would have returned) or an error.

Python

import time

while True:
    b = requests.get(f"https://mume.ai/api/v1/batches/{batch['id']}",
                     headers={"Authorization": f"Bearer {MUME_API_KEY}"}).json()
    if b["status"] in ("completed", "failed", "expired", "cancelled"):
        break
    time.sleep(60)

answers = {}
for item in b.get("results", []):
    if item.get("response"):
        answers[item["custom_id"]] = (
            item["response"]["body"]["choices"][0]["message"]["content"])
    else:
        print("failed:", item["custom_id"], item["error"])

print(b["usage"])   # {"prompt_tokens": …, "completion_tokens": …, "cost": …}

GET /api/v1/batches/:id/results returns just the results list, and answers 409 batch_not_completed until they exist. GET /api/v1/batches lists your batches, newest first. DELETE /api/v1/batches/:id removes a finished batch and its results; a running one cannot be deleted.

Which models

Every chat model with a :batch twin in the catalogue: the GPT-6 Astra, Sol, Terra and Luna families, Claude, Gemini, GLM, Kimi, Mistral, Qwen and more. The twin's page shows the batch price; the live model's page shows the live one. Sending a :batch id to a live endpoint answers 400 batch_only_model with the pointer here.

Limits

At most 1,000 requests per batch. Split larger jobs.
Image inputs must be public http(s) URLs; base64 and data: images are rejected in batch.
stream cannot be set on a request in a batch.