Pre-Prompt, Post-Prompt, Automate: A Mental Model for Working with AI
Most people, when they sit down to work with an AI, just open the chat box and start typing. The prompt is treated as the work itself. I have come to see it differently. The prompt is somewhere in the middle of the work, not the beginning of it.
Over the last couple of years of using these tools seriously in fraud and chargeback systems, I have settled on a simple three-stage model that I keep coming back to. Pre-prompt work. Post-prompt work. Automation. Three stages, in that order, and skipping any one of them is where most of my own mistakes have come from.
Stage 1: Pre-Prompt Work — Do The Homework First
Before I type a single word into the model, I sit down and write out what I already know. This sounds strange the first time you do it. Why write things down for yourself when you are the one with the knowledge?
The thing is, the AI does not have your years of operational knowledge. It has the public internet, a lot of books, a lot of code, and a lot of conversations. It does not have the fact that a particular acquirer flags certain BIN ranges differently than the others. It does not know that your chargeback team has a private heuristic about velocity in the first ten minutes of a new account. It does not know which fraud is the kind you actually want to stop, and which is the kind your business has quietly decided to accept as the cost of doing business.
So before the prompt, I do the knowledge transfer. What patterns have I seen? What edge cases am I already watching for? What did we try in 2019 that did not work, and why? I put it all in front of me. The prompt is then no longer a question. It is a briefing for a new collaborator.
Simon Willison has been making this same point for some time on his blog — the bottleneck with these tools is almost never the model itself. It is the context. Once you accept that, the homework is no longer optional.
Stage 2: Post-Prompt Work — Name Every Iteration
The first response from a model is almost never the final answer. The real work is the back-and-forth that follows. This is where, in my experience, most people lose the plot. They iterate without structure. Twenty messages later, they have a better answer but no idea how they actually got there.
This is why I name every meaningful iteration. Each improvement pass gets an ID — v1, v2, v3, and so on. Each one records three small things: what was the hypothesis, what changed in the prompt or in the approach, and what was the measured outcome.
Why bother with the naming? Because six iterations later, when something is finally working, I want to be able to point and say: it was v3 that gave us the unlock. And when something quietly starts getting worse — which does happen — I can also say: v5 made things measurably worse on the false positive rate, kindly revert. Without names, it is all one big undifferentiated conversation, and you cannot do real engineering on a soup.
Andrej Karpathy has written for a while that working with these models is starting to look more like data work than traditional coding (his site is karpathy.ai, well worth a read). I agree with this. And data work needs version numbers. Anthropic's own guidance on prompt iteration says much the same in different words — write your eval, run it, change one thing, run it again. The version-naming is just the personal, lightweight form of that same discipline.
Stage 3: Automation — Only When The Process Earns It
When the iteration improvements have stopped giving meaningful gains — when v12 and v13 look basically the same as v11 — only then does the process earn the right to be automated.
By automation I mean lifting the work out of the chat window altogether. It becomes a prompt template inside a script. It becomes a pipeline step. It becomes an agent that runs on a schedule, with the conversational scaffolding stripped away and the working core set in place.
The mistake people make at this stage is automating too early. Some months back, I almost did this myself with a chargeback narrative classifier. It was working, yes, but it was not yet stable. We had only done v2 at the time. If we had automated then, we would have frozen a process that still had its biggest improvements ahead of it. Automation is good and proper, but it must be the reward for a process that has stopped improving — not the rescue for a process you are simply tired of iterating on.
Why Bother With A Model At All
The whole thing is just three words: Pre. Post. Automate. It is not a complicated framework. Honestly, it is not really a framework at all. It is a way of slowing myself down at exactly the points where I am tempted to rush.
The pre-prompt step makes me do the homework I would rather skip. The post-prompt naming makes me track what I would rather forget. The automation gate makes me wait when I would rather just ship the thing. The model itself is small. The discipline it imposes is the entire point.