LLM Provider Adapters: Contracts for Streaming, Tools, and Fallback OnTech Insight

TECH INSIGHT / EXPLAINED

Adding a second LLM changes more than a URL. Streaming APIs differ in tool-argument assembly, completion signals, and usage semantics. An adapter should centralize those contracts instead of scattering them across callers.

AI-generated conceptual illustration. Three engineers unify different connectors through an adapter and validate response fragments.

Local execution of the actual project

These captures show the existing project UI or functions executed locally with non-private sample inputs. Each caption states the scope of the execution.

The original GoClaw provider-management screen running locally with a sample Ollama-type entry and its enabled state. This is a UI example; no model-server call, streaming, or fallback was executed.

Separate the common contract from optional features

The internal representation needs messages, tool calls, completion state, usage, and preserved provider metadata. Do not assume every model and endpoint supports streaming, images, and tools. Reject missing required capabilities explicitly; degrade only optional presentation features under a defined policy.

Document each adapter's supported models, endpoints, and unsupported capabilities. Run contract tests before enabling an integration. Separating serialization from transport makes anonymized response fixtures sufficient for many parser checks.

SSE is framed by events, not individual data lines

Standard SSE uses a blank line to delimit an event and joins multiple data lines with newlines. The following input represents one event with data first line\nsecond line. HTTP read-chunk boundaries are also independent of event boundaries.

event: example
data: first line
data: second line

The blank line after the final data line terminates the event. Preserve that trailing blank line when copying this block; EOF alone does not dispatch an unfinished event.

Let framing assemble an event, then let the provider adapter interpret it. Claude streams include a message_stop event. A closed connection alone is not proof of successful completion: it can also indicate a truncated stream. Do not discard completion, error, or usage events just because they contain no display text.

Validate tool arguments after assembly

JSON fragments can span events, and fragments for different tools may interleave. Assemble by tool-call ID and block index, then validate JSON and schema after completion. Arbitrarily changing original IDs can break the next tool-result association; preserve mappings to internal identifiers.

With Go's bufio.Scanner, handle token-size limits and scanner errors. Set bounded event size, total response size, and read time rather than unlimited buffers. Exceeding a limit must produce an explicit incomplete result.

Replacement and fallback are policy decisions

Falling back from tenant credentials to shared credentials changes billing and data processing. Allow it only by policy and record the provider and paying scope. Do not immediately close a client still used by active requests when swapping a registry entry. Use reference tracking or graceful retirement, and avoid slow Close operations under a global lock.

Start with five regression fixtures

Test multiline data, a UTF-8 character split between reads, interleaved tool fragments, EOF without a completion event, and usage arriving after text. These cases expose failures where visible text looks correct but the final response is wrong.

References

Content and examples reviewed: 2026-09-29

LLM Provider Adapters: Contracts for Streaming, Tools, and Fallback