Being Long On AI

The riskiest product decisions in AI rarely look risky when you make them.

Models are improving rapidly, but they’re still imperfect. They miss edge cases. They sometimes fail to follow instructions. Faced with that reality, teams do what good engineers have always done - they design around the limitations. This results in products based on decision trees and sub-agents, rigid workflows and deterministic fallbacks, and guardrails everywhere.

I get it. In many cases, taking this approach is what makes a system usable in the short term. It reduces risk, increases predictability, and helps teams ship something that works today. But there’s a distinction here that I think the industry is still underestimating - and it matters hugely when it comes to AI.

Compensating for Models vs. Baking Their Limits Into Your Product

Not all “pragmatic” decisions are the same. Some decisions acknowledge current model limitations while keeping the architecture flexible. Others quietly bake those limitations into the foundation of the system itself. The second category is where real technical debt starts to accumulate.

In AI, the most expensive technical debt isn’t messy code or missing abstractions. It’s architectural assumptions that only make sense if models don’t get significantly better.

In investing, being “long” on something simply means believing in its long-term trajectory. Most people in AI would say they’re long on model improvement. The interesting question isn’t whether you believe that, it’s whether your product architecture actually reflects it.

Designing for Model Improvement

Designing for model improvement means accepting real short-term pain. Early implementations are harder. You spend more time wrestling with edge cases, like writing crazy ‘YOU MUST’ prompts when the model doesn’t follow instructions. You sweat blood to make systems work when the models aren’t quite there yet. But you do that work once, rather than repeatedly rewriting your architecture every time models cross another capability threshold.

The alternative feels safer early on. You deliver faster. You control behavior more tightly. But over time, those safety mechanisms harden into constraints. When models improve - as they consistently have - your system starts fighting its own design. To take the example above, if you added deterministic behavior instead of fighting the prompt with ‘you must,’ once the models improve, you would be unable to benefit. Your agents wouldn’t be able to be fully autonomous because you would have hamstrung your infra with determinism.

Those same constraints often show up at the model boundary. When you design around the quirks of OpenAI or Anthropic or any other model, you’re not just compensating for today’s behavior - you’re encoding an assumption about which model you’ll be using. Switching providers down the line stops being a configuration choice and starts looking like a rewrite.

In practice, this tension shows up in very concrete product decisions, as it did for us, from how we chose to build our agents, to betting on fully-generative vs. deterministic workflows, and ReAg vs. RAG.

Dynamic Skills Instead of Sub-Agents

A common pattern in agent systems until recently was to split behavior across multiple sub-agents. A year ago this was the mainstream approach. The reasoning was straightforward: models couldn’t reliably decide which capability to use, so you routed requests deterministically.

We made a different assumption. Most of what people call “agent logic” actually lives inside the model. The difference between many sub-agents is often just the prompt. If a model is capable of reasoning, it should be able to load and unload the right instructions dynamically based on context.

But when we made this decision, that assumption wasn’t fully true yet. Models struggled with instruction following and context management. Making this work required a huge amount of engineering effort - it would have been easier to use orchestration layers and sub-agent hierarchies.

But doing so would have introduced permanent complexity for what we believed was a temporary limitation. So we built our agent architecture on dynamic skills, and as models improved, that bet paid off. The architecture didn’t need to be unwound. It simply became easier to operate, and dynamic skills have become the industry standard.

That’s what designing for model improvement looks like in practice: doing the hard thing early so you don’t have to undo the “safe” thing later.

Refusing the Determinism of Decision Trees

Decision trees feel enterprise-friendly. They look auditable and controllable, and give teams the sense that behavior is fully specced out with no surprises.

The problem is that humans don’t operate this way. People don’t follow decision trees. They reason. They adapt. They handle ambiguity. Deterministic workflows are a workaround for agents that can’t be trusted to do that yet.

Choosing not to encode that workaround into the product makes early systems harder to build. Getting agents to follow complex instructions reliably is genuinely difficult, especially when models are less capable.

But that scaffolding comes at a cost. Once you design autonomy out of the system, adding it back later is extremely hard. You end up with agents that can only handle the simplest cases - and break down as soon as real-world complexity shows up.

Betting Early on ReAG

A similar dynamic showed up with retrieval.

For a long time, traditional RAG approaches were considered safer. They were deterministic. They avoided hallucinations. LLM-based reasoning over large contexts was expensive and, early on, imperfect.

Relying on ReAG came with real costs: higher compute, more engineering work to manage accuracy and performance, and a willingness to tolerate early imperfections. We had to run multiple LLMs concurrently, and then run another one just to summarize the findings across all of them. But the underlying assumption was straightforward: context windows would grow, reasoning would improve, costs would fall, and accuracy would increase.

That’s exactly what happened. Because the architecture was already aligned with where models were heading, there was no painful migration later. No legacy retrieval logic to unwind. The system simply benefited from the improvement curve.

How AI Products Should Be Evaluated

The broader implication here isn’t about specific techniques. It’s about how we evaluate AI systems altogether. A useful question to ask of any AI product today isn’t just “what does it do?” but “what assumptions does it make about model limitations?”

Which parts of the system exist purely to compensate for today’s weaknesses? Which of those parts would become unnecessary, or actively harmful, if models got meaningfully better? Products that look stable early on often age poorly. Products that feel rougher early but are aligned with model trajectories tend to compound in strength.

The Cost of Playing It Safe

In AI, “playing it safe” often means baking yesterday’s constraints into today’s architecture. That can buy short-term confidence, but can also create long-term fragility.

The hardest part of building AI products right now isn’t integrating the latest model. It’s having the conviction to design as if improvement is inevitable, and the discipline to avoid locking in assumptions you already suspect won’t hold. The idea that models will keep getting better isn’t new. What’s still rare is designing systems as if that belief is actually true.

That tradeoff is uncomfortable. But in my experience, it’s the only one that allows you to succeed long term.