What Happens When You Talk to AIClearly Explained

Your answer was not found, it was finished

You type a question. A second later an answer appears. And most of us quietly assume it found that answer somewhere. It didn't.

No internet was read. No database was opened. Nothing was searched. Your answer was not found.

It was finished. Written one small piece at a time by a model that had never seen your question before. In the next seven minutes I'll show you exactly what happens between the keystroke and the answer. Why the same question twice gives two different answers, and the one number that decides how much the model can see.

Your message becomes tokens

First, your message gets chopped up before the model reads anything. The pieces are called tokens, and a token is the smallest unit a language model works with. It can be a whole word, part of a word, a character, even a byte.

Anthropic's documentation says that on earlier Claude models one token is roughly three and a half English characters, and that newer models use a newer tokenizer producing about thirty percent more tokens for exactly the same text. So token counts are not the same everywhere. Anthropic even ships a free endpoint that counts the tokens in your prompt before you send it, so you know roughly what it will cost.

The prediction loop

Now the model does the same thing over and over: it looks at everything so far and picks the next piece. Anthropic's glossary says it plainly: Claude is an autoregressive model, pretrained to predict the next word given the previous context. It is a machine for completing a pattern. Your message, the system prompt and the conversation so far are one long pattern, and each choice is conditioned on everything that came before.

Which is why the prompt matters so much: change one word at the top and you change the odds of every word after it. And it is why a model can write you something that never existed before. A search engine can only hand you a document that already existed. Here is the part that surprises people.

It does not think one word at a time

The model writes one word at a time, but it does not think one word at a time. Anthropic's interpretability team looked inside the model to check this, and found Claude plans many words ahead and then writes its way to that destination. In their words, even though models are trained to output one word at a time, they may think on much longer horizons to do so.

In one study they expected no planning and found the opposite: it planned a rhyme many words before it needed one. So the pause when you hit send is not one step. It is enormous computation for every word. So where did that pattern come from?

Where the pattern came from, and the training cutoff

Two stages. Stage one is pretraining: the model reads an enormous amount of text and guesses what comes next, over and over. Round after round, and every wrong guess nudges it a little. Stage two is refinement.

Anthropic's glossary calls it R L H F, reinforcement learning from human feedback: full answers get ranked, and the model is pushed toward the preferred ones. Every model has a training cutoff, the last date its training data covers, and Anthropic publishes them. Claude Opus five point five was trained on data up to June twenty twenty six. Sonnet five, up to January.

Past that date it does not reliably know. Web search is the tool for that, and every answer it produces comes back with citations. So if you ask about something that happened last week and it never mentions searching, treat that answer as a guess.

One desk, one window: the context window

Now the useful part. Everything the model can see lives in one place, and that place has a size. It is called the context window. Anthropic's docs call it a working memory.

The current Claude models have a one million token context window. On paid Claude plans the latest models match that; everything else sits at two hundred thousand, about five hundred pages. Everything counts: your system prompt, every message, uploaded documents, images, even the tool definitions. And the answer counts too.

The catch: Anthropic's own docs warn about something they call context rot. As the token count grows, accuracy and recall go down. A long conversation does not make the model smarter, it makes it vaguer. And chat apps manage the window on a rolling first in first out basis, so old turns quietly drop out.

Which is why a fresh chat for a new topic is not superstition. It is the fix.

The job card you never typed: the system prompt

There is one more thing at the top of that desk that you never typed: the system prompt. In the API it is a separate top level field, and Anthropic's reference says plainly that there is no system role inside the messages. It sits above your messages and applies to every turn. On Claude dot A I it also carries the current date and the house rules, which Anthropic updates over time.

Two things follow. First, this is where the job is defined. Anthropic's prompting guide says a role in the system prompt focuses behavior and tone, and that even a single sentence makes a difference. Second, it is a lever you can pull.

Add one line that says under forty words and no health claims, and the same question comes back a different way. Now the part that makes people suspicious.

Why two chats give two answers

Ask the same question in two chats and you get two answers, sometimes with different facts. Here is why, and it is not a bug. There is a setting called temperature, and it controls how much randomness goes into the prediction. Zero is most predictable, one is most varied, and the default is one.

Even at temperature zero, Anthropic's docs are clear: results are not fully deterministic, and identical inputs can produce different outputs. And they added a note recently: for models released after Claude Opus four point six, temperature is deprecated, so you cannot set it. The API accepts one point zero for backwards compatibility and rejects anything else. So the randomness is not a dial you control, it is a property of the model.

Two answers are normal. If the second one is better, keep that one.

Confident and wrong

And one more thing: the reason to keep your judgement. The model produces what a good answer would look like. Usually that lines up with reality. Occasionally it does not, and it says it with total confidence.

Anthropic's researchers found something sharper. They showed that Claude will sometimes produce a plausible sounding argument designed to agree with you rather than follow the logic. They literally caught it making up its own reasoning. Anthropic's own definition of an honest model gives accurate information, does not hallucinate, and acknowledges its limits when it should.

So a polished, confident answer is not proof. It is a pattern that completed the way patterns complete. Check it against a source before you act.

Four moves that change the result

Okay, four moves that change the result. One, give it the job: who you are, what you are building, what a good answer looks like. All of that becomes part of the pattern. Two, put the rules in the system prompt, not in a message from an hour ago.

It applies to every turn. Three, ask for options instead of one answer. There is no stored answer to retrieve, so ask for two. Four, know the date: if the answer depends on something recent, ask it to search, then read the citations.

That is the whole thing. Your message becomes tokens. The model predicts the next one, conditioned on everything before it. Everything it can see sits in one window, and that window has a size.

A system prompt tells it what job to do. None of that makes it deterministic, or always right. If this helped, subscribe. Build with AI.

For real.