Your AI Tool is a Service
In February I wrote that AI models are commodities and the moat is the human who knows what to ship and what to kill. That ending was right, but it was also the easiest part of the argument. “The human is the moat” is what you say when you’ve ruled out all the other answers. It doesn’t tell anyone what to actually build.
This week gave me the sharpest example I’ve seen of why the question matters.
Anthropic shipped Claude Opus 4.7 on Wednesday. For the five or six weeks leading up to it, Claude users — myself included — had been complaining that 4.6 felt noticeably worse: slower to reach a conclusion, more confidently wrong when it got there, conversations going sideways in ways they hadn’t a month earlier. The speculation filled in the gaps. Anthropic had quietly quantized the model to cut compute costs. Or they were diverting capacity to something more profitable, and the rest of us were eating the scraps.
The truth, as usual, was more boring. It came out after an independent analysis of 6,852 Claude Code sessions, published in early April by an AMD engineering director, measured a 67% drop in model reasoning since February. Anthropic’s response came within hours. Boris Cherny, who leads Claude Code, posted that two product changes had compounded over two months: in February, adaptive thinking was turned on by default, letting the model decide how much reasoning to do per turn. In March, the default effort level dropped from high to medium. A bug in the adaptive-thinking logic made it worse, with some turns getting zero reasoning tokens and the model hallucinating as a result. The weights didn’t change. The settings governing how much the model thinks before answering did.
That distinction is the whole point.
The weights didn’t change.
The settings governing how much the model thinks before answering did.
What This Week Was Actually Evidence Of
The model you’re using today is going to be worse than what’s available in six months, or quietly regress under you when you’re not looking, and you won’t know which it was until after it happens. Both pressures are real. Both are outside your control. Models are perishable.
If you run an AI product, this is the substrate risk. You built something that works. A new model ships and it’s 40% better on benchmarks. Great. Except your prompts were tuned to the old model’s quirks, your eval suite scored against its specific failure modes, and half the workarounds in your application exist because of behaviors the new model has already fixed. The upgrade is not free. The “commodity” claim starts to feel thinner.
Then the other direction: the model silently regresses for six weeks. Your users notice. You don’t have the telemetry to tell whether the problem is your product or the substrate, so you ship a patch that doesn’t help, because the problem was never yours.
In February I said the human is the moat. That’s true. But if I’m advising a product team after a week like this, “hire smart PMs” isn’t an answer… it’s a platitude. The real question is where human judgment actually persists when the model underneath keeps moving.
Where Judgment Freezes
Judgment compounds in the scaffolding around the model, because scaffolding is where human decisions get frozen into code, data, and interfaces. It’s not a separate moat. It’s where the moat lives.
Four layers, in rough order of how durable they are:
Retrieval. What candidates does the model even see? For any AI product pointed at a domain — search, support, sales, research — the quality of retrieval is the ceiling on output quality. The model can only be as good as the material it’s handed. Retrieval logic is frozen PM judgment: which fields matter, what ranking signals compound, what to surface when the user is vague. Swap GPT-4 for Opus 4.7 and that logic persists verbatim.
Domain grounding. What does a specialized term actually mean inside your company’s vocabulary? An LLM will give you a plausible answer, but it doesn’t know what your specific customers search for, how your analysts categorize incidents, or which queries should map to which internal concept until you’ve told it. Your taxonomy, your entity resolution, your disambiguation rules — these are the accumulated judgment of everyone who’s mapped your domain to machine-readable structure. They outlast every model you’ll build on.
Constraints. The model doesn’t know which customers are on which contract tier, which recommendations would violate compliance, which outputs are off-limits based on region or regulation or an existing commitment. Business rules live in application logic because they have to — not for reliability alone, but because they’re the one layer you can’t let the model guess at. Swap the underlying model and your constraints keep enforcing what can and can’t ship. Let the model enforce them and every version change becomes a compliance review. It’s the least glamorous layer and the one that decides whether your product can ship to enterprise at all.
Evaluation. This is the layer most teams skip, and it’s the one that tells you whether a model swap improved or regressed anything. An eval suite doesn’t have to be fancy. It’s a set of real queries from your actual users — not synthetic tests, not the benchmark suite the model vendor publishes — paired with expected output shapes and a scoring function you trust enough to act on. Twenty well-chosen queries beat two hundred generic ones. When a new model ships, you run the suite and look at what got better, what got worse, and what stayed laterally different. Without that, you’re relying on vibes and vibes scale badly when the substrate is perishable.
None of these are novel. They’re the boring parts of AI product work. They’re also where the judgment goes when a PM decides what their product should actually do.
Counterarguments, Compressed
Three objections worth handling directly.
Frontier capability still matters. True. Frontier-class capability isn’t commoditized — it’s restricted precisely because it’s too capable to release widely. The frontier is a real game. Most product teams aren’t playing it, and shouldn’t pretend to. Playing costs a GPU budget none of us have. Within any given tier, models commoditize fast. The strategic bet for a product team isn’t “our model is better than theirs.” It’s “our stack absorbs the next tier the moment it’s available.” That’s a different competition.
Fine-tuning is a moat. It’s a tactic, not a moat. Custom fine-tunes on proprietary data genuinely help, especially in narrow domains (legal, medical, specialized enterprise) where base models lack the patterns you need. But three things work against fine-tuning as durable advantage. Base-model updates routinely break them — you re-tune, or you lose the improvement. The ROI curve narrows every time base models get better at instruction-following, which has been steadily. And the maintenance cost compounds as your underlying data evolves. Fine-tuning is useful, expensive, and perishable. The scaffolding layer isn’t.
Model lock-in is real. Prompts, tool schemas, and eval harnesses do couple to specific models. Switching isn’t free. But “switching cost” and “moat” are different things. If the thing making you sticky to your customers is that you’re stuck with your model vendor, you built on sand. Good scaffolding is model-agnostic by design: the retrieval layer, the domain logic, the constraints all survive a model swap. The coupling lives at the prompt and tool-definition layer, which is the shallowest part of the stack.
Fine-tuning is useful, expensive, and perishable. The scaffolding layer isn't.
The Concrete Version
Over the last year I’ve kept evolving a personal setup: my Obsidian vault plus Claude Code, scaffolded with rules, commands, hooks, and agents I’ve built up over time. The underlying model has changed multiple times — Claude 3.5 Sonnet, Opus 4.5, 4.6, now 4.7. Each swap shifted behavior. One model got more verbose about reasoning through multi-step problems. Another silently changed how strictly it followed custom instructions. A third got sharper at analysis and slower on one-shot lookups. None of these changes were wrong. They were different defaults baked into different training runs, announced in blog posts nobody reads carefully enough.
What didn’t change: the rules that capture how I think — direct, no fluff, tables for comparisons, plans before implementation. The hooks that validate every note Claude writes, blocking anything with missing frontmatter or stale dates, regardless of which model is driving. The commands for recurring work (meeting synthesis, content review, scheduled summaries) that run the same way whether the underlying model changed last week or last year. The eval suite I run against critical workflows before letting a new model near them.
In one swap, the new model reverted to diplomatic-sandwich patterns I’d specifically ruled out in the style guide months earlier — “fair point, and...” openers in meeting notes, symmetrical paragraph structure in drafts. The rule existed because an older model kept doing exactly that; a newer one had apparently unlearned the correction. Without the audit step in the workflow, a handful of those would have gone out to stakeholders before I caught the drift.
That’s what frozen judgment looks like. Boring from the outside. The part of the setup that compounded over months of work, and what I’d still have if someone shipped something better tomorrow.
Which is happening right now. 4.7 shipped yesterday, and the community is already sorting into two groups — users reporting genuine regressions (long-context recall collapsed from 78% to 32% on Anthropic’s own benchmarks, rule-following drift in CLAUDE.md workflows, a new tokenizer that silently costs ~25% more per token), and users whose scaffolding caught the regressions before they hit. The second group is smaller and much quieter. It’s the group you want to be in.
What You Actually Own
You can’t invest in an abstraction. You can invest in the concrete artifacts where human judgment gets frozen: the taxonomy, the retrieval, the constraints, the evals. Those are what survive model swaps.
Models are rentals. The scaffolding is what you own. And the scaffolding only compounds if someone is paying attention to what it should actually do.
You rent the service. The judgment you’ve frozen into the scaffolding is what you own. That person is the moat. The stack is where they live.
I used AI tools throughout the research and drafting of this piece. Every claim was verified against primary sources. Every sentence was reviewed before publishing.