Rate limits
What is limited, what happens when you cross it, and how to find out before your users do.
Limits apply per organization, not per key — minting more keys does not buy more throughput.
What is limited
- Requests per minute, across all keys in the organization.
- Tokens per minute, counted as prompt plus completion.
Both vary by model: the small fast models carry the highest ceilings, the reasoning models the lowest, roughly in proportion to what a request costs the provider.
Crossing one
A 429 with a Retry-After header, in seconds. Honour it. Retrying sooner
extends the window rather than shortening it.
HTTP/1.1 429 Too Many Requests
Retry-After: 12
Staying under
- Cap
max_tokens. Token limits are usually what you hit first, and an unbounded completion is an unbounded share of the budget. - Put the cheap model in front. Routing, classification and moderation do not need the expensive one, and they are usually most of the volume.
- Spread retries. A thundering herd of clients all backing off by the same fixed interval arrives together.
Bringing your own
A provider key of your own uses your provider's limits instead of the shared pool's. If you have negotiated headroom, this is how you use it.