Deep Read, learning

How AI Models Actually Learn

Training builds the model once at huge cost. Inference answers cheaply, forever. Get the split and these tools stop feeling like magic.

The Curio Team · 5 min readWritten with AI from the sources below and checked against them before publishing.

Two very different things happen inside an AI model, and we think mixing them up costs people both money and clarity. Training is how the model gets built. Inference is what happens every time you send it a prompt. One is a giant, one-time construction project that runs for weeks. The other is a fast lookup you repeat all day. This single split explains most of what feels strange about these tools, from the price on your bill to the reason a model cannot truly remember your last chat. So let us pull the two apart.¹

Training builds the model once

A large language model is, underneath, a huge pile of numbers called parameters. Modern frontier models hold tens or hundreds of billions of them. At the start, those numbers are basically random. The model knows nothing. Training is the long, expensive process of nudging every one of those billions of numbers until the whole system reliably predicts the next word in ordinary human text.¹

Here is the loop, and it is worth picturing slowly. The model reads a chunk of text, tries to guess what comes next, and checks its guess against the real answer. When it is wrong, a signal flows backward through the network telling each parameter which direction to move to be a little less wrong next time. Then it does this again. And again. Trillions of times, across a dataset that spans much of the public internet, plus books and code. We are not exaggerating the scale. The finished parameters encode patterns nobody wrote down by hand.⁴

Gradient descent in plain English

The math that steers those nudges is called gradient descent, and the picture is simpler than the name. Imagine we are standing on a foggy hillside and want to reach the lowest point in the valley. We cannot see far at all. So we feel the slope under our feet and take one step downhill. Then we feel again and step again. The gradient is just that slope, the direction of steepest descent, computed for every parameter at once.³

The valley here is error. Height above the valley floor is how wrong the model currently is, measured by something called a loss function. Each training step measures the slope of that error and takes a small step down it. Do this enough times and the billions of parameters settle into a configuration that captures deep structure in language. Nobody hand-writes those patterns. They emerge, slowly, from millions of tiny steps down the same foggy hill.³

Why training costs a fortune and inference does not

Training is brutally expensive because you must run that backward slope calculation across every parameter, for trillions of words, on thousands of specialized chips working in parallel for weeks or months. It is a one-time capital cost, like building a factory. Once it finishes, the parameters are frozen. That frozen file is the model you and everyone else then use, unchanged, for the rest of its life.⁴

Inference is what happens when you actually prompt a finished model. Now there is no backward pass and no learning at all. The numbers do not move. Your text flows forward through the frozen network once, and out comes a prediction for the next token, then the next, then the next. That is dramatically cheaper, which is why providers charge per million tokens rather than per training run, and why one answer often costs a fraction of a cent even though training the same model cost millions of dollars.²

The model we chat with is frozen glass, not wet clay. It answers, but it never learns from us.

This gap has a consequence people feel constantly. The model does not remember your previous session, because remembering would mean rewriting its parameters, and inference never touches them. Anything it seems to recall inside one conversation is simply text being fed back into that same forward pass as context. Close the tab and that context is gone.

What this means for how you use these tools

Once the training and inference split clicks, a lot of practical decisions get easier. We stop expecting the model to improve because we corrected it yesterday. We start feeding it exactly what it needs today, right there in the prompt, on every single call.

A few habits follow directly from the frozen-model reality:

• Treat every chat as a fresh start. The model holds no memory of you across sessions, so paste the context, the examples, and the constraints it needs each time rather than assuming it kept them.
• Pay for capability, not for teaching. When you buy access, you are renting inference on a model someone already trained. Prompting well, adding retrieval, or supplying your own documents moves results far more than hoping the model quietly studied you overnight.
• Match the model to the job. Bigger models cost more per token at inference, so use a strong one for hard reasoning and a smaller, cheaper one for bulk work like tagging or summarizing, where the extra parameters buy you almost nothing.
• Watch the token meter, not the training story. Your cost scales with how many tokens go in and how many come back on every call, so trimming bloated prompts and capping output length is the most direct lever you have.

None of this requires math. It requires one mental model: the thing was shaped once at enormous cost, and it now answers cheaply and repeatedly, and never changes again. Hold that picture and these tools stop feeling like magic. They start feeling like what they truly are, which is a frozen, very well-tuned prediction machine that we rent by the token.