Designing CLI Tools for LLM Agents

🌐 한국어

When you hand tools to an LLM agent, tools designed for humans cause problems. Humans like tabular output and colors; agents need consistent, machine-parseable structure. From building a trading CLI that cron calls periodically and an LLM evaluates, here are the design principles.

Principle 1 — All Output Is ok/error JSON

Whether the tool succeeds or fails, output schema is uniform. The agent checks one ok field and branches:

# Success

{ "ok": true,  "data": { ... } }

# Failure

{ "ok": false, "error": "...", "exception_type": "...", "traceback": "..." }

Extract this boilerplate into a common wrapper so you don't repeat it in every script:

import json, sys, traceback

def print_json(data):

    print(json.dumps({"ok": True, "data": data}, ensure_ascii=False, default=str))

def print_error(message, exc=None):

    payload = {"ok": False, "error": message}

    if exc is not None:

        payload["exception_type"] = type(exc).__name__

        payload["traceback"] = traceback.format_exc()

    print(json.dumps(payload, ensure_ascii=False, default=str))

    sys.exit(1)  # signal failure via exit code too

def run(handler):

    try:

        print_json(handler())

    except SystemExit:

        raise

    except Exception as e:

        print_error(str(e), exc=e)

Each tool is just logic wrapped by run(). Success = exit 0 + ok:true, failure = exit 1 + ok:false. Works for agents and shell scripts:

One real trap: on Windows console (cp949 encoding), printing Korean or emoji crashes with UnicodeEncodeError. Reconfigure stdout to UTF-8 at startup:

sys.stdout.reconfigure(encoding="utf-8")

Principle 2 — State Snapshots, Not History

Call your agent on cron every tick, and naively you'd log everything. But then context explodes, and the LLM gets whipsawed by old logs.

Instead, overwrite a single state file—snapshot, not history:

def _load_state():

    if not STATE_FILE.exists():

        return {"schema_version": 3, "positions": [], "last_trade_ids": {}}

    return json.loads(STATE_FILE.read_text(encoding="utf-8"))

def _save_state(state):

    STATE_FILE.write_text(

        json.dumps(state, ensure_ascii=False, indent=2), encoding="utf-8")

Append to a ledger file (ledger.jsonl) only for events that truly matter (fills, settlements). Snapshot overwrites; history accumulates—separate concerns:

Most importantly, the source of truth is the external system, not the state file. For trading: current positions and open orders always come fresh from the exchange, not from state. The state file holds only what the exchange doesn't know (last measured value, agent's choice). This way, even if state corrupts, the system stays coherent.

Principle 3 — Single Entry Point for Each Cron Tick

Don't make the agent figure out which tool to call first. Provide one entry point (tick) that gathers all raw data for judgment in one call:

# tick.py — single entry point for each cron tick (query only)

# No rules or decisions here. Just data; LLM decides.

def build_tick():

    return {

        "account":   fetch_account(),    # balance / available / unrealized PnL

        "positions": fetch_positions(),  # exchange is the source

        "macro":     measure_macro(),    # raw macro indicators

        "signals":   measure_symbols(),  # per-symbol indicators + delta vs last

    }

run(build_tick)

Here's the critical design choice: keep trading logic out of the tick tool. You'll be tempted to embed "if this condition, enter" rules into code, but then the agent becomes a shell and the code is the strategy. Keep tick as pure data provider, give judgment entirely to the LLM. Separate order execution (place_order, close_position) as distinct explicit tools; blur data and action at your peril.

Summary

• Output as single ok/error JSON schema with matching exit codes
• State snapshots (overwrite) + append-only ledger for true history
• Source of truth is the external system, re-queried each tick
• Cron calls one entry point; logic lives in the LLM prompt, not code

Design tools as thin data and action layers, and you swap strategies by rewriting the prompt, not the code. That's the goal of agent-friendly CLI.

Autonomous agent trading carries hallucination risk. Test thoroughly in small scale and sandboxes. This is technical documentation, not investment advice.