One Interface for 20 LLM Providers: The Adapter Pattern in Practice
Build a product on an LLM and you will add a second provider. Because of pricing, because a particular model does something better, or because the first one had an outage. And that's the moment you discover that "send messages, get a reply" is the same everywhere while the way it's expressed is different everywhere.
Some take the system prompt as a top-level field; others want it inside the message array. Tool results are shaped differently. Streaming event names differ. Some signal end-of-stream explicitly and some don't. Let those differences leak into application code and every new provider adds another if provider == "...".
This article is about confining that variation to an adapter layer.
1. Make the Shared Interface Brutally Small
The most common mistake is a large interface. Keep saying "this provider supports X, let's add a method" and you end up with an interface where 3 of 20 implementations do something meaningful and the other 17 return nil.
Include only what every implementation can genuinely do. In practice, four methods.
type Provider interface {
// Send messages, get a response
Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error)
// Streaming — emit chunks via callback, return the assembled response at the end
ChatStream(ctx context.Context, req ChatRequest, onChunk func(StreamChunk)) (*ChatResponse, error)
DefaultModel() string
Name() string
}The important detail is that ChatStream streams via callback and still returns the final response. Callers need chunks for live output and simultaneously need the assembled result — tool calls, usage. Providing both from one method removes all the chunk-reassembly code from call sites.
2. Handle Capability Differences with Optional Interfaces
So what about extended features? In Go, optional interfaces are the answer. Define them separately and check with a type assertion where needed.
// Implemented only by providers supporting extended reasoning
type ThinkingCapable interface {
SupportsThinking() bool
}
// Implemented only by providers holding per-session external state
// (long-lived child processes, dedicated connections)
type SessionCloser interface {
CloseSession(ctx context.Context, sessionKey string) error
}
// Call site: use it if supported, move on quietly otherwise
if tc, ok := provider.(ThinkingCapable); ok && tc.SupportsThinking() {
req.Options[OptThinkingLevel] = level
}The benefit is that adding a new capability touches zero existing implementations. Adding a method to the main interface means editing 20 files; a new optional interface only needs implementing in the one place that supports it.
3. Declare Static Capabilities as a Struct
For capabilities you branch on at runtime, a declarative struct beats a method. Better to ask once and hold the value than to run ten type assertions to learn "does this stream?"
type ProviderCapabilities struct {
Streaming bool // supports ChatStream?
ToolCalling bool // accepts tool definitions?
StreamWithTools bool // can stream while tool calls are in flight?
Thinking bool // supports extended reasoning?
Vision bool // accepts image inputs?
CacheControl bool // supports block-level cache directives?
MaxContextWindow int // context window for the default model
TokenizerID string // which tokenizer to use for counting
}
type CapabilitiesAware interface {
Capabilities() ProviderCapabilities
}StreamWithTools sitting apart from Streaming is a scar from production. Implementations exist that stream fine until tool calls enter the mix and then emit broken chunks. A combination constraint like that can't be expressed by one boolean, so it gets its own field.
4. Separate Serialization from Transport
Inside each provider implementation, split one more layer. Converting between internal and provider formats and actually firing the HTTP request are different concerns.
// Handles only the translation between internal and provider-specific formats.
// Composed inside each Provider implementation (it does not replace Provider).
type ProviderAdapter interface {
// internal request -> provider wire format (bytes + headers)
ToRequest(req ChatRequest) ([]byte, http.Header, error)
// provider response bytes -> internal response
FromResponse(data []byte) (*ChatResponse, error)
// one SSE chunk -> internal chunk. Returns nil for chunks to skip.
FromStreamChunk(data []byte) (*StreamChunk, error)
Capabilities() ProviderCapabilities
Name() string
}Split this way, the translation logic is testable without HTTP. Capture real response bytes from a provider into a file, feed them to FromResponse, and verify the parse. No network, no mock server. Most of the time you spend adding a provider goes into this translation code, so making it easy to test makes a large difference.
It also matters that FromStreamChunk can return nil. SSE streams carry keep-alives, metadata, and start/stop signals that contain no actual content. Surface those upward as "empty chunks" and every caller has to filter them. Swallowing them as nil in the adapter is correct.
5. Absorbing SSE Differences — What Actually Varies
Streaming is where providers diverge most. Both use SSE (Server-Sent Events), but the details differ.
- OpenAI-style: only
data:lines. Sends an explicit terminator,data: [DONE], at the end of the stream. - Anthropic-style: announces an event type first via
event:lines (message_start,content_block_delta, and so on). There's no[DONE]marker; the stream ends when the connection closes.
The common mistake here is writing a fresh SSE parser per provider. You don't need to. One scanner handles both formats.
// Reads an SSE stream line by line and extracts data payloads.
// Tracks both event: types and data: payloads so several providers can share it.
func (s *SSEScanner) Next() bool {
for s.scanner.Scan() {
line := s.scanner.Text()
// Track event type (used by Anthropic-style streams)
if after, ok := strings.CutPrefix(line, "event:"); ok {
s.eventType = strings.TrimSpace(after)
continue
}
// Extract the data payload
var payload string
if after, ok := strings.CutPrefix(line, "data: "); ok {
payload = after
} else if after, ok := strings.CutPrefix(line, "data:"); ok {
payload = after
} else {
continue // skip blank lines, comments, other fields
}
// [DONE] is the OpenAI-style terminator
if payload == "[DONE]" {
return false
}
s.data = payload
return true
}
s.err = s.scanner.Err()
return false
}The shared scanner handles only the line-level protocol; interpreting the payload belongs to each adapter's FromStreamChunk. The concerns separate cleanly.
One production trap: raise the scanner buffer size. Go's bufio.Scanner defaults to a 64KB maximum line. Responses carrying images or long tool call arguments exceed it and the stream dies silently. That bug is extremely hard to track down.
sc := bufio.NewScanner(r)
sc.Buffer(make([]byte, 0, initBufSize), maxBufSize) // the 64KB default isn't enough6. A Registry — Look Up by Name, Fall Back When Missing
With many implementations you need a lookup point. Adding hierarchical fallback there pays off in multi-tenant setups: use the provider a tenant registered with their own key if it exists, otherwise drop to the shared configuration.
type Registry struct {
providers map[string]Provider // keyed "tenant/name"
mu sync.RWMutex
}
func (r *Registry) GetForTenant(tenantID uuid.UUID, name string) (Provider, error) {
r.mu.RLock()
defer r.mu.RUnlock()
// 1) look for a tenant-specific registration first
if tenantID != MasterTenantID {
if p, ok := r.providers[key(tenantID, name)]; ok {
return p, nil
}
}
// 2) otherwise fall back to the shared default
if p, ok := r.providers[key(MasterTenantID, name)]; ok {
return p, nil
}
return nil, fmt.Errorf("provider not found: %s", name)
}Don't forget to close the previous instance on replacement. Implementations holding HTTP clients, connection pools, or long-lived processes leak quietly otherwise.
func (r *Registry) Register(tenantID uuid.UUID, p Provider) {
r.mu.Lock()
defer r.mu.Unlock()
k := key(tenantID, p.Name())
if old, ok := r.providers[k]; ok {
if c, ok := old.(io.Closer); ok {
c.Close() // clean up before replacing — prevents resource leaks
}
}
r.providers[k] = p
}What You Actually Run Into Adding a New Provider
- Tool schema dialects — the same JSON Schema is supported to different depths. Some reject
oneOf, some require the top level to be anobject, some 400 on specific keywords. A schema normalization layer eventually becomes necessary. - Tool call ID rules — some implementations return a 400 when IDs repeat across iterations of an agent loop. Mix in the iteration index to keep them unique.
- Passing reasoning blocks back — models using extended reasoning sometimes require the reasoning blocks, with their signatures, returned verbatim on the next request. Reserve a field in your response struct to preserve that raw content.
- Proxies claiming "compatible" APIs — they frequently 400 on unknown fields. Send optional fields only to native endpoints.
Summary
- The shared interface holds only what every implementation truly does — four methods is enough
- Extended features go in optional interfaces so new capabilities don't touch existing implementations
- Expose runtime-branching capabilities as a declarative struct
- Separating translation from transport makes parsing testable without HTTP
- SSE should share line-level parsing and vary only in interpretation — and raise that buffer size
- The registry gets hierarchical fallback plus cleanup on replacement
The point of the adapter pattern isn't elegance, it's isolating change. When one provider alters its API, do you edit one file or twenty? That difference is all that's really left.
댓글
댓글 쓰기