Insights / AI
Designing an API boundary for AI features
A model should not become an untyped, over-privileged shortcut through your application. Put authorization, validation, and failure handling at the boundary.
Kiran Bandarupalli · 2 Oct 2026 · 2 min read
It is tempting to let a browser call a model provider directly or to pass a large application object into a prompt helper. Both choices blur an important boundary: the model is an external dependency, not an authority inside the product.
Keep the model behind a narrow contract
Expose an application operation such as draftSummary(recordId), not a generic endpoint that accepts arbitrary prompts and provider options. The server should authenticate the caller, check access to the record, select only the data needed for the task, and enforce input limits before making a provider request.
Represent the result with a typed contract. If the feature expects a short summary and source references, validate that shape after generation. Treat missing fields, invalid identifiers, unexpected tool calls, and oversized output as ordinary failure cases—not as values to cast into the application's trusted domain model.
Make failure behavior explicit
Model calls can time out, be rate-limited, return malformed data, or be unavailable. Set a bounded timeout and retry only failures that are safe to retry. Give each user action an idempotency key if a retry might create a durable object. Return a clear state to the UI, and preserve the user's original work when generation fails.
Authorization and data minimization
Apply access checks before retrieval and again before any consequential write. Do not rely on instructions in a prompt to protect private records. Send the smallest useful context, redact secrets and unrelated identifiers, and decide how prompts and responses are retained before adding verbose logging.
Operational controls
- Cap input and output size, request frequency, and concurrent work.
- Record provider, model, prompt version, latency, and outcome without logging sensitive content by default.
- Keep model selection behind configuration so a provider change does not reshape the whole application.
- Test timeout, refusal, malformed output, permission denial, and provider outage paths.
A good AI boundary makes the product easier to secure and easier to evolve. It also keeps the model's uncertainty visible: the application can show a suggestion, ask for review, or decline to act instead of quietly treating generated text as fact.