Writing

Monday Question: How Many Calls Does It Make?

Before you ship an AI feature, ask this about it:

How many model calls does one request actually make?

Not roughly. The actual number, and whether it's fixed or it depends on what the input turns out to be. If you can't answer without checking the code, that's worth noticing, because it usually means the feature was built by trial and error rather than designed.

One call is a simple feature: extract, classify, summarize, done. Two or three in a fixed sequence is still a pipeline you can reason about. An unbounded number that depends on what the model decides to do next is a different thing entirely, and it needs to be built and priced like one, not discovered like one after the invoice arrives.

This isn't a technical detail for the engineers to worry about. It's the actual cost and reliability profile of the feature. The distinction between a fixed pipeline and something agent-shaped is exactly this question, and it's the same discipline behind the pipeline we run for our own products.

Ask it about whatever AI feature shipped most recently. If nobody can answer immediately, that's the finding.


This is usually the first thing we check when reviewing someone else's AI feature. Get in touch if you want a second set of eyes on yours.