API Rate Limits: Shared Quotas, Token Buckets and Safe Retry Decisions

한국어로 읽기

TECH INSIGHT / EXPLAINED

A rate limiter controls when a request may be attempted. A retry policy separately decides whether repeating that operation is safe. Mixing the two can turn a transient timeout into a duplicate order.

AI-generated educational illustration. Three colleagues use a token bucket, waiting timer and priority queue to schedule API requests.

Match the quota the server actually counts

Identify the quota key: IP, account, API key, endpoint group or request weight. A process-local bucket does not coordinate several bot instances sharing that quota. Reserve scheduler capacity for reconciliation and cancellation instead of letting background reports consume everything.

A token bucket with refill r and capacity B allows a burst as well as a long-run rate. It does not guarantee “at most N in every one-second window.” Choose the model and burst against the server’s actual contract; extra local buckets never create extra server quota.

Go implementation and cancellation

golang.org/x/time/rate supports Allow, Reserve and Wait; WaitN can account for weighted requests within the configured burst. Pass a context deadline. Include the waiting time in the total request budget and use cancellation-aware timers for retries, rather than an unconditional sleep after shutdown.

For a safe read, use bounded exponential backoff with jitter. Set maximum attempts and a total deadline. Do not sleep again after the final attempt; return the last meaningful error with context.

Retry-After has two formats

An HTTP Retry-After value can be seconds or an HTTP date. Parse both. Treat it as the earliest retry time: if it exceeds the operation deadline, return a deferred/failure result rather than silently truncating the server’s wait and retrying too early.

from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
def retry_after_seconds(value, now):
    if not value:
        return None
    value = value.strip()
    if value.isascii() and value.isdigit():
        return float(int(value))
    try:
        target = parsedate_to_datetime(value)
        if target.tzinfo is None:
            return None
        return max(0.0, (target-now).total_seconds())
    except (TypeError, ValueError, OverflowError):
        return None
now = datetime(2026, 9, 29, 0, 0, tzinfo=timezone.utc)
assert retry_after_seconds('5', now) == 5
assert retry_after_seconds('Tue, 29 Sep 2026 00:00:10 GMT', now) == 10

A timeout after an order is not a retry instruction

A read timeout or some 5xx responses may occur after the order was accepted. For order creation, reconcile using the venue’s documented client ID/idempotency behavior before any repeat. Some IDs only assist lookup and do not guarantee idempotency. Permanent parameter errors need correction, not more attempts.

Monitor throttled counts, quota remaining where available, queue age and reconciliation delay. Self-throttling reduces avoidable violations but cannot promise that a shared quota will never be exhausted.

Sources and further reading

Go rate limiter · HTTP Retry-After · Requests error-handling companion

Content and examples reviewed: 2026-09-29