Everyone uses it. Almost nobody can explain what it actually is.
The usual explanations don't help. Revolutionary. Human-like intelligence. None of that tells you what's happening when you type something and get an answer back.
There are two phases, and mixing them up is where the confusion starts.
Phase one: training. This already happened.
Before a model could talk to you, it read an enormous amount of text. Books, articles, code, conversations. From that, it learned patterns. Which words follow which words. What a good answer usually looks like.
All of that got compressed into billions of numbers, a giant table of weights. This happened once, long before you opened a chat window. The model isn't learning anymore after this point.
Phase two: prediction. This is what happens when you talk to it.
When you type something, the model isn't learning from you. It's using what it learned during training to predict what text comes next, one token at a time, until it decides it's done.
It doesn't think. It doesn't reason. It predicts, based on patterns it already has.
This explains a lot:
- Why a model can write something completely wrong, with total confidence. It's predicting what a correct answer sounds like, not verifying facts.
- Why it has a knowledge cutoff. Training stopped at some point. Anything after that doesn't exist for the model, unless you give it that info yourself.
- Why the same question phrased differently gets a different answer. You're changing which patterns it matches against.
- Why more context makes it better. You're handing it more material to predict from, not making it think harder.
Not a search engine
Most people treat an LLM like a search engine with better writing. Ask a question, get the facts.
That's not what's happening. A search engine finds something that already exists. An LLM generates something that resembles what it learned. Most of the time that's genuinely useful. Sometimes it's a confident, well-written, completely wrong paragraph.
I've built real products on top of this. The day training versus prediction actually clicked, I changed how I work with these models. I stopped trusting them with calculations. I stopped assuming confidence meant correctness. I started treating every output as a strong first draft, never a verified answer.
Next week: why AI lies about numbers, and what to do instead.