AI-Native Architecture

1 Probabilistic Behavior

Traditional components are deterministic. Give them the same input and you get the same output.

That property is so ordinary that most of us have never said it out loud, and an enormous amount is built on top of it. Unit tests. Regression suites. Canary comparisons. The entire practice of reproducing a bug before fixing it.

Put a model in the call path and the assumption is gone. Nothing warns you.

An AI-native architecture treats this as a design constraint rather than a surprise.

• Evaluation replaces assertion. A test compares output to an expected value and returns a boolean. An evaluation runs a set of representative cases, scores each against a definition of acceptable, and reports a rate. Pass and fail stop being properties of a single run.
• The eval set comes from production. Cases the builders imagined will always score better than cases users actually send. The gap between those two sets is where the failures live.
• Somebody outside engineering sets the threshold. Ninety percent acceptable is either fine or a catastrophe depending entirely on what the output touches, and that is not an engineering call.
• Verification is a component in the system. If something cannot guarantee its own output, something else has to check it. That checker sits in the request path, costs money, takes time, and can fail on its own. Those are architectural properties and they need architectural decisions.
• Low confidence is a branch. Most models emit a confidence signal and most systems throw it away. A component that can say it does not know gives the architecture somewhere else to send the request.

Full article: It Passed the Test. That Doesn’t Mean It Works. — publishing September 1

2 Inference Economics

A traditional function call costs microseconds and effectively nothing. An inference call costs real money and real latency, and both vary with model size, context length, and load.

This is the same shift that happened when we moved to distributed systems. Once a call crossing a network became categorically different from a call that did not, a whole set of patterns followed. Batching. Caching. Circuit breakers. Locality. None of those were optimizations added afterward. They became part of how systems were designed.

What is new is that the cost is partly chosen rather than suffered. Model size is a choice. Context length is a choice. Both are made in code and both appear on an invoice thirty days later.

• Routing is architecture. Which model handles which call belongs in the design, with a name and a place on the diagram. Sorting a request into one of twelve categories is not the same work as composing a reply to an angry customer, and it should not cost the same.
• Routing requires thresholds. “Is the cheaper model good enough here?” has no answer without an acceptance threshold, which is why property one comes first.
• Caching on meaning is a correctness decision. Exact-match caching barely helps, because two people asking the same thing produce different strings. Semantic caching works, and it means the system has decided two different questions deserve the same answer. How close is close enough is a threshold somebody owns.
• Per-transaction cost has to be instrumented in. If nobody can say what one transaction costs, nobody can say whether the feature is worth what it costs.
• Inference cost moves without a deploy. A longer prompt, a retrieval step returning more context, a shift in what customers ask. A team can double its spend without shipping anything.

Full article: Your Function Call Was Free. This One Isn’t. — publishing September 8

3 The Context Supply Chain

AI components are only as good as the context you feed them. That makes data pipelines, retrieval systems, and freshness guarantees first-class architectural concerns rather than back-office plumbing.

Garbage in, garbage out is old news. What changed is the output. Bad input used to produce output that looked bad, and a person glancing at the screen caught it. Feed a model stale or wrong context and it produces something fluent, organized, and confident.

Confidence became free. Accuracy did not.

• Every context source needs a contract. Where does this content come from, down to the system and the team that owns it? How current does it have to be? What is this component permitted to see? What happens when retrieval comes back empty?
• Freshness is a correctness bound with a number and an owner. Pricing and refund policy tolerate almost no lag. An internal wiki tolerates a lot. The behavior at the boundary has to be specified, because the default behavior of every component involved is to keep answering.
• The dependency has to be visible in both directions. If a system reads a content source, the people who edit that source need to know it, and a change needs to be an event the system can react to. Otherwise a correct edit by a person doing their job well silently makes the system wrong.
• The retrieval layer is a security perimeter. A model with access to the wrong context is one cleverly worded question away from a data breach. Least privilege assumes a structured request and a permission check at a boundary you control. Retrieval has neither.
• Content provenance is transitive dependency management. Vendor PDFs, licensed research, customer submissions. Each carries terms, and almost none of those terms were written with “a model will read this and paraphrase it to strangers” in mind.

Full article: The Model Isn’t Wrong. Your Context Is. — publishing September 15

4 Model Lifecycle

Services version on your schedule. Models drift, get deprecated, and improve on someone else’s.

Architects have always managed dependencies we do not control, and the usual answers do not quite fit. Pinned model versions exist, but they carry no behavioral guarantee. A model contract covers the shape of the response and the availability of the endpoint, and says nothing about whether the answers stay useful. Deprecation windows are set by the vendor.

And there is a problem the old playbook has no name for. With a library, “improved” and “broken” are different words. With a model, a real improvement can be indistinguishable from a regression for a system tuned around the previous behavior.

• The model goes behind a boundary. Prompt construction, model selection, retry behavior, and response parsing live on your side of an interface. What crosses is your types.
• The evaluation harness is the actual control. The boundary lets you swap. The harness tells you what the swap did. A boundary without a harness creates confidence without information.
• Drift and deprecation need different responses. Deprecation is scheduled and visible, so it is a project. Drift is continuous and quiet, so it needs a review cadence rather than a trigger.
• Portability is a decision somebody has to make. The provider-neutral abstraction layer is one of the most reflexively built and least examined pieces of architecture in this area. It is frequently money spent on optionality nobody will exercise. The question worth asking is what you would actually do differently if your provider changed something tomorrow.

Full article: You Didn’t Change Anything. It Changed Anyway. — publishing September 22