> ## Documentation Index > Fetch the complete documentation index at: https://docs.zerogpu.ai/llms.txt > Use this file to discover all available pages before exploring further. # Production patterns > What to get right when you move ZeroGPU from a first call into production. Keep API keys in environment variables, never in source or client-side code. Handle the common status codes explicitly: 401 (bad API key), 403 (bad project ID), and 429 (rate limit). Set sensible timeouts and retries with backoff on 429 and 5xx. For large, non-real-time jobs, use the Batch and Files API instead of the synchronous endpoint - streaming is not supported in batch mode. Already on the OpenAI SDK? The drop-in client shown in the [Introduction](/) is the recommended way to call ZeroGPU from application code. ## Keep secrets out of source Read your `x-api-key` and `x-project-id` from the environment (or a secrets manager) and inject them at deploy time. Never commit them, never ship them in a browser bundle or mobile app - a key embedded in client-side code is a public key. ZeroGPU calls authenticate from your backend. If you need to call from a browser or mobile client, proxy the request through a server you control so the key stays server-side. ```bash theme={null} # .env (git-ignored) - load with your process manager or dotenv export ZEROGPU_API_KEY="zgpu-..." export ZEROGPU_PROJECT_ID="..." ``` ## Handle status codes explicitly Branch on the status code. Authentication and authorization errors are permanent - retrying them just burns time and quota. Rate limits and server errors are transient - those are the ones to retry. | Status | Meaning | What to do | | ------ | ---------------------- | --------------------------------------------------------------------------------- | | `200` | Success | Parse and use the response. | | `400` | Bad request | Fix the request body. Do **not** retry. | | `401` | Bad API key | Check `x-api-key`. Do **not** retry. | | `403` | Bad project ID | Check `x-project-id` and permissions. Do **not** retry. | | `420` | Input over token limit | Shorten the input. Do **not** retry unchanged. | | `429` | Rate limit reached | Wait at least `Retry-After` seconds, then retry. See [Rate limits](#rate-limits). | | `5xx` | Server error | Retry with exponential backoff. | Treat `408` (request timeout) and `409` (conflict) the same as `5xx` for retry purposes. Network errors and client-side timeouts are retriable too. ## Rate limits Your organization has limits for each model, shared by all of its API keys: * **Requests per minute** * **Tokens per minute** - counted when a request is accepted: the estimated input tokens plus `max_tokens` (`max_completion_tokens`, or `max_output_tokens` on the Responses API) for each of `n` choices. Tokens a response doesn't use are not given back, so set `max_tokens` close to the size of the response you expect. * **Concurrent requests** - requests still in progress, including open streams. Counts reset at the start of each minute. Audio endpoints have request and concurrency limits, but no token limit. Responses carry your limits in headers. A header is left out when that limit doesn't apply. | Header | Example | Meaning | | ------------------------------------------- | -------- | ------------------------------------ | | `x-ratelimit-limit-requests` | `300` | Requests allowed per minute | | `x-ratelimit-remaining-requests` | `295` | Requests left this minute | | `x-ratelimit-reset-requests` | `37s` | Time until the request count resets | | `x-ratelimit-limit-tokens` | `500000` | Tokens allowed per minute | | `x-ratelimit-remaining-tokens` | `499000` | Tokens left this minute | | `x-ratelimit-reset-tokens` | `37s` | Time until the token count resets | | `x-ratelimit-limit-concurrent-requests` | `16` | Requests allowed in progress at once | | `x-ratelimit-remaining-concurrent-requests` | `15` | Requests you can start right now | A request over a limit is rejected with `429` before it runs, and `error.code` says which limit it hit: ```json theme={null} { "error": { "message": "Rate limit reached for deepseek-v4-flash in organization 3f2c... on requests per min (RPM): Limit 300, Used 300, Requested 1. Please try again in 37s.", "type": "rate_limit_error", "param": null, "code": "requests_per_minute_exceeded" } } ``` | `error.code` | What happened | What to do | | ------------------------------ | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `requests_per_minute_exceeded` | Too many requests this minute | Wait for `Retry-After`, then retry. | | `tokens_per_minute_exceeded` | Too many tokens this minute | Wait for `Retry-After`, then retry. If there is no `Retry-After`, this request alone is over your limit: reduce the input or `max_tokens`. | | `concurrency_limit_exceeded` | Too many requests in progress | Retry after `Retry-After`, or when one of your requests finishes. | Rejected requests still count toward your requests per minute, so retrying in a tight loop keeps you limited. Batch jobs use the same limits as your synchronous calls. To raise a limit, contact support. ## Set timeouts and retries Three rules cover almost every case: 1. **Set a per-request timeout** so a stalled connection can't hang your worker. 2. **Retry only the transient codes** (`408`, `429`, `5xx`) and network failures - never `401`, `403`, or `400`. 3. **Back off exponentially with jitter**, cap the delay, cap the attempts, and honor the `Retry-After` header on `429`. ### Using the OpenAI SDK (recommended) If you call ZeroGPU through the [drop-in OpenAI client](/), timeouts and retries are built in - set `timeout` and `max_retries` once on the client. The SDK retries `408`, `409`, `429`, and `5xx` with exponential backoff and respects `Retry-After` automatically. ```python Python wrap theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.zerogpu.ai/v1", api_key="unused", # ZeroGPU authenticates via the headers below default_headers={ "x-api-key": os.environ["ZEROGPU_API_KEY"], "x-project-id": os.environ["ZEROGPU_PROJECT_ID"], }, timeout=30.0, # seconds, per request max_retries=5, # exponential backoff on 408 / 409 / 429 / 5xx ) resp = client.responses.create( model="llama-3.1-8b-instruct-fast", input="Your input text here...", ) print(resp.output) ``` ```javascript JavaScript wrap theme={null} import OpenAI from 'openai'; const client = new OpenAI({ baseURL: 'https://api.zerogpu.ai/v1', apiKey: 'unused', // ZeroGPU authenticates via the headers below defaultHeaders: { 'x-api-key': process.env.ZEROGPU_API_KEY, 'x-project-id': process.env.ZEROGPU_PROJECT_ID, }, timeout: 30_000, // ms, per request maxRetries: 5, // exponential backoff on 408 / 409 / 429 / 5xx }); const resp = await client.responses.create({ model: 'llama-3.1-8b-instruct-fast', input: 'Your input text here...', }); console.log(resp.output); ``` You can override either value per request, for example a longer timeout on a heavy call: `client.responses.create(..., timeout=60.0)`. ### Rolling your own When you call the HTTP API directly, implement the loop yourself: a per-request timeout, a retriable-status check, and exponential backoff with jitter that honors `Retry-After`. ```bash cURL wrap theme={null} # curl's built-in --retry handles 408/429/500/502/503/504 with exponential # backoff and honors Retry-After. It does NOT retry 401/403/400 - exactly right. curl --retry 5 --retry-delay 1 --max-time 30 \ https://api.zerogpu.ai/v1/responses \ -H "content-type: application/json" \ -H "x-api-key: $ZEROGPU_API_KEY" \ -H "x-project-id: $ZEROGPU_PROJECT_ID" \ -d '{ "model": "llama-3.1-8b-instruct-fast", "input": "Your input text here..." }' ``` ```python Python wrap theme={null} import os import random import time import requests URL = "https://api.zerogpu.ai/v1/responses" RETRIABLE = {408, 429, 500, 502, 503, 504} HEADERS = { "content-type": "application/json", "x-api-key": os.environ["ZEROGPU_API_KEY"], "x-project-id": os.environ["ZEROGPU_PROJECT_ID"], } def create_response(payload, max_retries=5, timeout=30): for attempt in range(max_retries + 1): retry_after = None try: resp = requests.post(URL, json=payload, headers=HEADERS, timeout=timeout) if resp.status_code not in RETRIABLE: resp.raise_for_status() # raises on 401 / 403 / 400 - no retry return resp.json() if attempt == max_retries: resp.raise_for_status() retry_after = resp.headers.get("retry-after") except requests.exceptions.RequestException: if attempt == max_retries: raise # exponential backoff with jitter; honor Retry-After on 429 delay = float(retry_after) if retry_after else min(2 ** attempt, 30) + random.random() time.sleep(delay) ``` ```javascript JavaScript wrap theme={null} const URL = 'https://api.zerogpu.ai/v1/responses'; const RETRIABLE = new Set([408, 429, 500, 502, 503, 504]); const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); async function createResponse(payload, { maxRetries = 5, timeoutMs = 30_000 } = {}) { const headers = { 'content-type': 'application/json', 'x-api-key': process.env.ZEROGPU_API_KEY, 'x-project-id': process.env.ZEROGPU_PROJECT_ID, }; for (let attempt = 0; attempt <= maxRetries; attempt++) { const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), timeoutMs); try { const res = await fetch(URL, { method: 'POST', headers, body: JSON.stringify(payload), signal: controller.signal, }); if (!RETRIABLE.has(res.status)) { if (!res.ok) throw new Error(`ZeroGPU ${res.status}: ${await res.text()}`); // 401 / 403 / 400 return res.json(); } if (attempt === maxRetries) throw new Error(`ZeroGPU ${res.status} after ${maxRetries} retries`); // exponential backoff with jitter; honor Retry-After on 429 const retryAfter = res.headers.get('retry-after'); const delay = retryAfter ? Number(retryAfter) * 1000 : Math.min(2 ** attempt * 1000, 30_000) + Math.random() * 1000; await sleep(delay); } catch (err) { if (attempt === maxRetries) throw err; await sleep(Math.min(2 ** attempt * 1000, 30_000) + Math.random() * 1000); } finally { clearTimeout(timer); } } } ``` ```rust Rust wrap theme={null} use reqwest::Client; use serde_json::Value; use std::time::Duration; use tokio::time::sleep; const RETRIABLE: [u16; 6] = [408, 429, 500, 502, 503, 504]; async fn create_response(payload: Value, max_retries: u32) -> Result> { let client = Client::builder().timeout(Duration::from_secs(30)).build()?; let api_key = std::env::var("ZEROGPU_API_KEY")?; let project_id = std::env::var("ZEROGPU_PROJECT_ID")?; for attempt in 0..=max_retries { let mut retry_after: Option = None; match client .post("https://api.zerogpu.ai/v1/responses") .header("content-type", "application/json") .header("x-api-key", &api_key) .header("x-project-id", &project_id) .json(&payload) .send() .await { Ok(resp) => { let status = resp.status(); if !RETRIABLE.contains(&status.as_u16()) { if !status.is_success() { // 401 / 403 / 400 - permanent, do not retry return Err(format!("zerogpu {}: {}", status, resp.text().await?).into()); } return Ok(resp.json().await?); } if attempt == max_retries { return Err(format!("zerogpu {} after {} retries", status, max_retries).into()); } retry_after = resp .headers() .get("retry-after") .and_then(|v| v.to_str().ok()) .and_then(|v| v.parse().ok()); } Err(err) => { if attempt == max_retries { return Err(err.into()); } } } // exponential backoff; honor Retry-After on 429 let backoff = retry_after.unwrap_or_else(|| 2u64.pow(attempt).min(30)); sleep(Duration::from_secs(backoff)).await; } unreachable!() } ``` ```go Go wrap theme={null} package main import ( "bytes" "encoding/json" "fmt" "io" "math" "math/rand" "net/http" "os" "strconv" "time" ) var retriable = map[int]bool{408: true, 429: true, 500: true, 502: true, 503: true, 504: true} func createResponse(payload map[string]any, maxRetries int) ([]byte, error) { body, _ := json.Marshal(payload) client := &http.Client{Timeout: 30 * time.Second} for attempt := 0; ; attempt++ { req, _ := http.NewRequest("POST", "https://api.zerogpu.ai/v1/responses", bytes.NewReader(body)) req.Header.Set("content-type", "application/json") req.Header.Set("x-api-key", os.Getenv("ZEROGPU_API_KEY")) req.Header.Set("x-project-id", os.Getenv("ZEROGPU_PROJECT_ID")) res, err := client.Do(req) if err == nil && !retriable[res.StatusCode] { defer res.Body.Close() data, _ := io.ReadAll(res.Body) if res.StatusCode >= 400 { // 401 / 403 / 400 - permanent return nil, fmt.Errorf("zerogpu %d: %s", res.StatusCode, data) } return data, nil } if attempt >= maxRetries { if err != nil { return nil, err } return nil, fmt.Errorf("zerogpu %d after %d retries", res.StatusCode, maxRetries) } // exponential backoff with jitter; honor Retry-After on 429 delay := time.Duration(math.Min(math.Pow(2, float64(attempt)), 30))*time.Second + time.Duration(rand.Intn(1000))*time.Millisecond if err == nil { if ra := res.Header.Get("Retry-After"); ra != "" { if secs, e := strconv.Atoi(ra); e == nil { delay = time.Duration(secs) * time.Second } } res.Body.Close() } time.Sleep(delay) } } ``` ```ruby Ruby wrap theme={null} require 'net/http' require 'uri' require 'json' ENDPOINT = URI('https://api.zerogpu.ai/v1/responses') RETRIABLE = [408, 429, 500, 502, 503, 504].freeze def create_response(payload, max_retries: 5, timeout: 30) http = Net::HTTP.new(ENDPOINT.host, ENDPOINT.port) http.use_ssl = true http.open_timeout = timeout http.read_timeout = timeout request = Net::HTTP::Post.new(ENDPOINT.request_uri) request['content-type'] = 'application/json' request['x-api-key'] = ENV.fetch('ZEROGPU_API_KEY') request['x-project-id'] = ENV.fetch('ZEROGPU_PROJECT_ID') request.body = payload.to_json (0..max_retries).each do |attempt| delay = begin response = http.request(request) status = response.code.to_i unless RETRIABLE.include?(status) raise "ZeroGPU #{status}: #{response.body}" if status >= 400 # 401 / 403 / 400 return JSON.parse(response.body) end raise "ZeroGPU #{status} after #{max_retries} retries" if attempt == max_retries # exponential backoff with jitter; honor Retry-After on 429 response['retry-after']&.to_f || [2**attempt, 30].min + rand rescue Net::OpenTimeout, Net::ReadTimeout raise if attempt == max_retries [2**attempt, 30].min + rand end sleep(delay) end end ``` ## Use the Batch API for large jobs For large, non-real-time workloads, the [Batch and Files API](/docs/batch) is the right tool instead of looping over the synchronous endpoint. It processes up to 50,000 requests within a 24-hour window at a discounted rate and sidesteps per-request rate limits entirely - so you don't need a retry loop at all. | You need… | Use | | --------------------------------------------------- | ------------------------------------------------------------------------- | | A single immediate response | The synchronous endpoint (`POST /v1/responses`) with the retry loop above | | Thousands of completions, can wait minutes-to-hours | The [Batch API](/docs/batch) | | To avoid per-second rate limits during a backfill | The [Batch API](/docs/batch) | | Streaming responses | The synchronous endpoint - streaming is **not** supported in batch mode | ## Next steps