Public Crypto Candle Data: Pagination, Gaps and Closed-Bar Archives
TECH INSIGHT / EXPLAINED
Public candle APIs are useful for learning and reproducible research. The collection task is more than downloading rows: record the source, page boundaries, timezone, missing intervals and whether the latest candle has closed.
Fetch a bounded page first
For Upbit Korea, GET /v1/candles/minutes/60 accepts market=KRW-BTC and count up to 200. Supported minute widths currently include 1,3,5,10,15,30,60 and 240. Use a timeout, HTTP/JSON validation and shared rate-limit coordination.
The to cursor selects candles before its timestamp. Use an explicit UTC timestamp, find the oldest returned candle and require each next cursor to move backward. Stop on an empty page, your date target or a page limit. Do not assume every response is sorted without checking it.
No trade can mean no candle
Upbit creates candles for intervals with trades, so a gap is not automatically a collection failure. Preserve that distinction. A policy that synthesizes empty bars changes indicator inputs and must be recorded; never silently fill a gap with an invented trade.
The normalizer below illustrates one-hour closed-candle selection and conflict detection. It is not a complete collector or OHLC validator. Actual price and volume fields still need finite/range checks before export.
from datetime import datetime, timedelta, timezone
def normalize_hourly(rows, market, now):
if now.tzinfo is None:
raise ValueError('aware now required')
unique = {}
for row in rows:
if row['market'] != market:
raise ValueError('market mismatch')
# This field is explicitly UTC in the API schema.
start = datetime.fromisoformat(row['candle_date_time_utc']).replace(tzinfo=timezone.utc)
if start + timedelta(hours=1) > now:
continue # exclude the still-open candle
if start in unique and unique[start] != row:
raise ValueError('conflicting duplicate candle')
unique[start] = dict(row)
return [(t, unique[t]) for t in sorted(unique)]
sample = {'market':'KRW-BTC','candle_date_time_utc':'2026-09-29T00:00:00','trade_price':100}
now = datetime(2026,9,29,1,30,tzinfo=timezone.utc)
assert len(normalize_hourly([sample,sample],'KRW-BTC',now)) == 1Keep exchange schemas separate
Binance Spot /api/v3/klines uses a different array schema and supports up to 1,000 rows per request. Its open time identifies the bar; interpret timestamp units and configured interval timezone according to that API. Do not mix its BTCUSDT prices with Upbit KRW-BTC as if they were the same currency series.
Keep decimal text in the archive where available, and convert deliberately for analysis. Sort and deduplicate by exchange, market, interval and UTC opening time. Duplicate pages are a condition to handle, not a claim that every exclusive-cursor response duplicates its boundary.
Make the archive reproducible
Export UTC timestamps with offsets, OHLCV, source venue, interval and collection time. Save request parameters and raw pages beside the normalized CSV. If two records with the same key disagree, preserve the evidence and resolve the revision rather than silently choosing one.
A fixed sleep is only local pacing and cannot guarantee compliance with shared quotas. Follow the endpoint’s current limits and access/redistribution terms; public availability does not imply unlimited history, unrestricted redistribution or sufficient detail to simulate order queues.
Sources and further reading
Upbit minute candles · Binance Spot klines · Upbit rate limits
Content and examples reviewed: 2026-09-29
댓글
댓글 쓰기