# If a New Model Drops Next Month, Does Your Business Get Better or Worse?

> AI models are retired on a schedule, sometimes with two months' notice. The businesses that gain from each new release own what the model can't supply: their rules, their expertise and the system built on them.

By Dom O'Carroll. Published 2026-10-23; facts last checked 2026-10-23. Canonical: https://subfractal.com.au/thinking/new-model-better-or-worse

## The short version

- A new model makes you better if what makes your AI valuable lives outside the model: the rules it works within, the expertise it draws on, the tests that tell you whether it's right, and a person who answers for the result. It makes you worse if your product is mostly someone else's model. Models are now retired on a schedule, so build on the parts you own.

Greg Isenberg recently described a dinner in San Francisco where someone asked a table of well-known founders, "If a new model drops next month, does your company get better or worse?" Half the table went quiet.

It's a good question for any business using AI, not just AI startups. Ask it about your own tools this week.

## How often do AI models change?

Constantly, and on the vendor's timetable, not yours. Anthropic's published policy is to give customers with active deployments at least 60 days' notice before a publicly released model is retired. After that date, requests to the old model fail.

That policy is real in practice:

- Anthropic notified customers on 14 April 2026 that two of its 2025 models would retire. They were switched off on 15 June 2026.
- On 30 September 2026 it gave notice that Claude Sonnet 4.5, released the previous September, retires on 30 November 2026.

Other vendors run similar lifecycles, and on some partner platforms the same model can retire on a different date.

None of this is a complaint. Each new model is usually better than the last. But if your process depends on how one particular model behaves, you have a change every few months that you didn't schedule, and as little as two months to deal with it.

## What gets worse when the model changes?

Anything whose value was tied to the model's quirks:

- **Long prompts tuned by trial and error** to coax one model into behaving. A new model reads them differently.
- **Workarounds for old weaknesses.** A process built to correct a model's errors can start "correcting" a model that no longer makes them.
- **Products that are mostly the model.** If what you sell is a thin layer over someone else's AI, each release makes the layer thinner, and makes it easier for a customer to go straight to the source.

## What gets better when the model changes?

Anything that exists independently of the model and gives it something to work with. Isenberg's notes describe the strongest teams building what he calls vertical harnesses for one industry. They contain four things: connectors to that industry's data, tests written by people who know the work, rules for what the agent may and may not touch, and a screen where a person reviews the output. In his estimate, the model itself is maybe 20% of the result.

We'd describe the same thing in the terms we build with:

1. **The rules.** What AI may do in your business, what it must never do, and who signs. Written down, outside any prompt.
2. **The expertise.** What your best people know and decide: which readings to distrust, which client asks for what, which judgement calls carry the business. Captured in a form a machine can use.
3. **The tests.** Real examples from your own work, with the right answer agreed by someone who knows. They are how you know, within a day, whether a new model is better or worse at your work rather than at a benchmark.
4. **The record.** What the AI got wrong and what a person changed. That is evidence for an auditor, and the best tuning data you will ever have.

Build an expert system on those four things and a better model makes it better, almost for free. The site says it more briefly: builds depreciate, understanding appreciates.

## How do you tell whether the AI is actually working?

Not with benchmarks. One founder Isenberg met judges every output by what users do with it: do they edit it, export it, or share it? For a business, the equivalent is simple. How much of what the AI drafts does a person change before it goes out, and is that number going down?

If nobody is measuring that, a model change is a coin toss. If someone is, a model change is just Tuesday.

## Five questions to ask your team this week

1. **Could we switch to a different model in a day?** If not, what would break?
2. **Do we have twenty real examples, with agreed right answers, that we could run a new model against?**
3. **Are our rules for AI written down outside the prompts?** Or do they exist only as instructions buried in a tool?
4. **Do we know which of our tools call which model, and who owns each?** (This is the AI register, and most businesses don't have one.)
5. **When the AI is wrong, who notices, and where is that written down?**

Three or more "no"s means a new model is a risk to you, not an opportunity. That's fixable, and it's mostly work about your business, not about AI.

## Sources

- [Greg Isenberg, "48 hours in SF", Greg's Letter, 22 October 2026 (observations 3 and 4, and the closing question)](https://www.gregisenberg.com/)
- [Anthropic, Model deprecations: notice policy, 2026 deprecation history, and the Sonnet 4.5 retirement on 30 November 2026](https://platform.claude.com/docs/en/about-claude/model-deprecations)
