Datassier Get in touch
Field note // Runtime discipline

Most of what you automated with AI makes you run out of tokens on a Thursday.

A client asked me which model each of his scheduled scripts should use. It is a reasonable question with a better one hiding behind it: why does any of them need a model?

He had built roughly fifteen recurring tasks over a few months. Inbox watchers, a morning briefing, reconciliation sweeps, a daily payments check. All of them useful, all of them working, all of them fired on a schedule by an AI assistant running under his personal subscription. By Tuesday afternoon he had consumed 45 percent of his weekly allowance. The week before, he ran dry on Friday and got it back on Sunday.

His instinct was to economise: assign a cheaper model to the simpler jobs. That instinct is correct and it fixes about a fifth of the problem. The rest of the problem is that most of those jobs are not thinking about anything.

Three categories, and only one of them needs a model

01

Triggers. Something needs to run at 06:00 and put its output somewhere. There is no judgment in this at all. A scheduler does it for nothing, and does it more reliably than a model, which can be asked to fire and simply not be there.

02

Deterministic work in costume. Pull these rows, fill this fixed template, check whether the earlier job produced a file. This looks like AI work because an AI was asked to do it. A file-existence check is one line of code, and unlike a model it cannot report success for a run that never happened.

03

Actual judgment. Read a document written by a human in a format nobody standardised. Decide whether this message is the thing you were waiting for. Propose a match and show the evidence. This is the category models are extraordinary at, and it is smaller than it looks.

When we walked his list, the third category was a handful of jobs. Everything else was a scheduler and some code wearing an expensive coat. The quota was not being spent on intelligence. It was being spent on orchestration, formatting, and re-reading the same rows every morning.

The floating engine

Then the detail that mattered more than the bill. His scheduled tasks ran on whatever model his desktop app happened to be set to when they fired. Leave it on the most capable model on Thursday night and Friday's run used that. Switch it Monday and Monday's run was a different system.

Read that again with money in mind. A job that posts payments to an accounting ledger had a different engine depending on what its owner was doing the night before. Nothing in the output would tell you which. Nothing in the logs would either.

If the version of the thing making your decisions can change without a commit, you do not have a system. You have a habit.

This is why production code pins its dependencies to exact versions, and models are a dependency like any other. Pin the model by its full identifier, in one config file, under version control, per task. Not because the newest model is worse, but because you should decide when the engine changes, and there should be a line in a diff saying so.

Kill, move, downgrade, in that order

The order is the whole point. Most people start at the third step because it is the one that feels like optimisation.

Kill: take the model out of everything with no judgment in it. This is usually half the volume and it costs nothing but an afternoon of plain code. Move: put what remains of the scheduled work on metered infrastructure with its own budget and credentials, so a production job can never be starved by a person exploring on the same allowance. Downgrade: now choose the smallest model that clears the bar for each remaining task, and prove it with a set of real inputs whose answers you already know.

Do it in the other direction and you save twenty percent and still hit the wall on Thursday.

The general form

Three times in a fortnight, across three different technologies, the same finding: a production system resting on a personal-scale foundation. An audit ledger in a consumer file-sync folder. An account-wide API key inside an application. Fifteen scheduled business jobs on one person's chat subscription.

Every one of them worked. That is what makes the pattern hard to see from the inside, and it is the same trap as the one described here: the failure mode of good prototypes is not breaking, it is growing. Nothing warns you on the day you add the fifteenth job. The warning arrives on a Thursday, when the thing that posts your cash stops running because you spent the morning building something else.

Scheduled work belongs on scheduled rails: its own budget, its own credentials, its own logs, its own pinned versions. Keep the model where the judgment is. Everything else is a cron job, and a cron job has never once run out of tokens.

Client details anonymised. If your automations grew faster than the ground under them, write to hello@datassier.com.