Debugging a Stuck Trading Bot: A Checklist

🌐 한국어

You open the dashboard in the morning and it's frozen at 3 PM yesterday. No error log, and the process is alive. Start scattering print statements at random in that situation and you lose a day. This article is about having an order to follow.

⚠️ This article is for educational and informational purposes. It is not investment advice, and you alone are responsible for any losses.

The First Fork

"It stopped" lumps together two completely different states. The diagnostic paths diverge entirely, so settle this first.

• A. It died — no process. Panic, OOM, system restart. The cause is almost always near the last log line.
• B. It's alive and doing nothing — the process exists but makes no progress. Blocking wait, deadlock, infinite loop, or a condition that keeps evaluating false. This one is much harder to find.

# Linux

ps -ef | grep mybot

# Windows PowerShell

Get-Process | Where-Object { $_.Name -like "*bot*" }

For A, the exit code and the last three log lines almost always give you the answer. For B, work down the list below.

Log Review Order — Top to Bottom

1. Timestamp of the last log — when did it stop? Check whether it coincides with a periodic boundary: market close, midnight, the top of the hour. If it does, you're already halfway to the cause.
2. The ten lines before it — what was the last successful action? Whether it stopped at "quote received" or at "order sent" completely changes the scope of everything that follows.
3. Is the same message repeating? — if one line keeps printing, it isn't dead, it's trapped in a loop. Code without retry backoff usually lands here.
4. Warning-level logs — there may be no errors but there may well be warnings. Filter for WARN alone.
5. External signals — exchange notices (maintenance, API changes), server reboot records, free disk space.

That's five minutes. Skipping those five minutes and diving straight into the code is the most common way to waste time.

Eight Common Causes

1. Expired Auth Token

The most frequent one. A one-day token expired past midnight and the refresh logic is missing or failed. The symptom is "every API call fails after a specific time," but if you're swallowing exceptions it just goes quiet. Check whether 401s or 403s are printing consecutively.

2. Blocking Read That Waits Forever

A WebSocket or socket read waits indefinitely, unable to distinguish a dropped connection from a silent peer. TCP keeps reporting "connected" for a while after the other side is gone. Without a timeout or watchdog, this is a permanent stop.

3. Rate Limit Block

You sent requests too often and your IP or key is temporarily blocked. A 429 makes it obvious, but some exchanges just delay the response instead. Then it's indistinguishable from a timeout.

4. Deadlock

You take a lock and take the same lock again, or two goroutines each wait on the lock the other holds. The form especially common in Go is making a network call while holding a lock. The moment the network slows, everything else stops.

// ❌ network under lock — a 30s response freezes everything for 30s

mu.Lock()

resp, _ := http.Get(url)

state.Update(resp)

mu.Unlock()

// ✅ network outside the lock

resp, _ := http.Get(url)

mu.Lock()

state.Update(resp)

mu.Unlock()

5. A Blocked Channel Halts Everything

Send on an unbuffered channel while the receiver is dead or slow and the sender stops right there. If a logging channel blocks this way, the entire bot halts over a single log line.

select {

case logCh <- msg:

default:            // if the receiver falls behind, drop this log line

}

6. Silently Swallowed Exceptions

The worst offender. Put except: pass—or Go's _ = err—inside a loop and it fails every cycle while appearing to be "operating normally," without a single log line.

# ❌ you will never know what happened

try:

    place_order(...)

except Exception:

    pass

# ✅ at least leave a trace

except Exception as e:

    log.exception("order failed: %s", e)   # stack trace included

7. A Condition That Stays False — the "Normal" Stop

Not a bug: a predicate keeps returning false so nothing happens. The classic case is a market-hours check thrown off by the server's timezone. Deployed to a cloud box running UTC, it computed Korean trading hours as the middle of the night and waited all day. Because it isn't an error, no alert fires either.

The remedy is logging. Record the skips, with reasons.

if not is_market_hours():

    log.debug("outside market hours — skipping (now=%s)", datetime.now(KST))

    return

8. Data Arrives but the Values Never Change

The connection is alive and messages keep coming, but the same value repeats. This happens when the upstream feed breaks. Monitoring connection state alone will never catch it—you have to look at the variety of the values.

closes = df["close"].tail(6).tolist()

if len(closes) >= 5 and len(set(closes)) <= 2:

    log.warning("suspected stale data — holding off on indicator calculation")

    return None

Memory leaks (a list that keeps growing over long runs until OOM) and a full disk belong to the same family. Both blow up days later, so cause and symptom sit far apart.

How to Reproduce It

Once you've narrowed the cause, reproduce it to confirm. You don't need to wait for a live session.

• Network drop — block the port briefly with a firewall rule, or turn off Wi-Fi. Watch how many seconds the bot takes to notice.
• Token expiry — force the token variable to an expired value, or set the expiry one minute out.
• Slow response — stand up a local proxy with a five-second delay, or inject time.sleep(5) for the test.
• Deadlock — in Go, SIGQUIT (kill -QUIT on Linux) dumps every goroutine's stack, showing you immediately which line is stuck. Python's equivalent is faulthandler.dump_traceback_later().

import faulthandler

faulthandler.dump_traceback_later(60, repeat=True)  # stack dump every 60 seconds

Making Next Time a Five-Minute Job — the Watchdog

This is the highest-return investment. Record the timestamp of the last message received and have a separate loop periodically check its age.

// refresh on every message received

atomic.StoreInt64(&lastRecvAt, time.Now().UnixMilli())

go func() {

    t := time.NewTicker(15 * time.Second)

    defer t.Stop()

    for range t.C {

        if !isMarketHours() {

            continue

        }

        age := time.Now().UnixMilli() - atomic.LoadInt64(&lastRecvAt)

        switch {

        case age > 60000:

            log.Printf("[WS] no data for 60s (%.1fs) — reconnecting", float64(age)/1000)

            conn.Close()      // force the blocking read awake

            return

        case age > 30000:

            log.Printf("[WS] warning: no data for 30s")

        }

    }

}()

There's one design point here. The watchdog doesn't reconnect itself. It only calls conn.Close() to wake the blocking read with an error, and lets the existing reconnect path do its job naturally. Not building two recovery paths is far safer.

Add three more things and most of the "why did it stop" questions disappear.

• Lifecycle logs — always record start and shutdown. Then the logs alone tell you "died or stalled."
• Heartbeat — even when nothing happens, write "alive + current state summary" every minute. The point where logs stop becomes the point where it stopped.
• Panic recovery — put a recover on every worker goroutine so one dying doesn't take the whole process down.

func safeGo(name string, fn func()) {

    go func() {

        defer func() {

            if r := recover(); r != nil {

                log.Printf("[PANIC] %s: %v", name, r)

                notify("worker panic: " + name)

            }

        }()

        fn()

    }()

}

That said, recover isn't a cure-all. Continuing to run with already-corrupted state can be worse, so always alert after recovery, and default to halting trading.

Summary

• First determine died vs stalled. The diagnostic paths differ
• Last log timestamp → previous ten lines → repetition → warnings → external factors, in five minutes
• Common causes: expired token · blocking read · rate limit · deadlock · blocked channel · swallowed exception · false condition · stale data
• Reproduce with network blocks, forced token expiry, injected delay, and stack dumps
• Prevent with watchdog + heartbeat + lifecycle logs + panic recovery

Real skill in bot operations isn't avoiding bugs—it's how few minutes it takes to pinpoint the cause when one hits. And shortening that time is mostly a matter of log design, not code.