How Does ChatGPT Work? LLMs Explained Simply

You type a question, and a second later ChatGPT replies with a neat, confident answer, as if a very well-read friend were on the other end. So how does ChatGPT work, really? Is it searching the internet? Is it thinking? The real answer is surprisingly simple to grasp, and in this guide you will see it for yourself, with two small hands-on demos you can play with right here on the page.

No maths degree needed. If you can use the autocomplete on your phone, you already understand the first big idea.

How does ChatGPT work? The one-sentence answer

Here it is: ChatGPT is a super-powered autocomplete.

When you type “See you” on your phone, the keyboard suggests “soon” or “later”. It has noticed which words tend to come next. ChatGPT does the same thing, but on a vastly bigger scale. Instead of looking at the last word or two, it considers your whole message (and the conversation so far), and instead of offering three suggestions, it weighs up every possible next piece of text.

It picks one piece, sticks it on the end, and asks itself the same question again: “Given everything so far, what comes next?” Do that hundreds of times in a row and you get a paragraph, an email, a poem or a working piece of code.

What is an LLM? Large language models explained

The engine inside ChatGPT is a large language model, or LLM. Other assistants, such as Claude or Gemini, are built on LLMs too. The name tells you a lot:

• Language: it works with text. It reads text in and writes text out.
• Model: a mathematical system that has learned patterns from examples, rather than being given a list of hand-written rules.
• Large: it has an enormous number of adjustable internal settings, called parameters, and it was trained on a huge amount of text.

Under the hood, an LLM is a kind of neural network: a big web of simple maths operations, loosely inspired by how brain cells connect. Each parameter is just a number. During training, those numbers are nudged again and again until the network becomes very good at one job: guessing the next bit of text.

The specific design most modern LLMs use is called a transformer. Its trick is a mechanism called attention, which lets the model look back across the whole text and decide which earlier words matter most for the next one. In “The trophy didn’t fit in the suitcase because it was too big”, attention helps the model link “it” to “trophy”.

Step 1: Your words become tokens

So, what are tokens in AI? Computers can’t read letters the way we do, so the first step is to chop your text into small chunks called tokens. A token might be a whole common word (“cat”), a piece of a longer word (“un” + “believ” + “ably”), a number, or a punctuation mark.

Each token is then swapped for a number, and those numbers are what the model actually works with. Tokens also explain a few everyday quirks: AI tools measure usage and limits in tokens, and models sometimes struggle with tasks like counting the letters in a word, because they never see individual letters, only chunks.

Try it yourself. Type a sentence, or a long, unusual word, and watch how it splits:

Step 2: Predicting the next token (how LLMs work)

Now the fun part. The model takes all the tokens so far and produces a score for every token it knows. Those scores are turned into probabilities: “mat” might get a big chance, “sofa” a smaller one, and “moon” a tiny one.

Then one token is chosen, added to the text, and the whole process runs again with the slightly longer text. That is why you see ChatGPT’s answer appear word by word: it really is being written one token at a time.

What “temperature” does

If the model always picked the single most likely token, its writing would be predictable and a bit dull. So instead it usually samples: it rolls a weighted dice, where likely tokens have more sides.

A setting called temperature controls how weighted that dice is. Low temperature makes the favourite almost certain, which is good for facts and code. High temperature evens the odds out, which gives more creative, surprising text, and eventually nonsense. This is also why the same question can get a different answer each time you ask.

Your turn to be the language model. Pick a starter, move the slider, and build a sentence:

Notice how, at low temperature, “Once upon a” almost always continues with “time”, while at high temperature you might get “Once upon a teapot”. Real models do exactly this, just with far richer odds that come from everything they learned.

How ChatGPT learned: training, fine-tuning and feedback

Nobody typed those probabilities in by hand. The model learned them. That happened in roughly three stages.

1. Pre-training: reading a library

First, the model is shown a huge amount of text, such as books, articles, websites and code. Over and over, it sees some text with the next token hidden, makes a guess, checks the real answer, and adjusts its parameters a tiny bit to do better next time.

Repeat that an astronomical number of times and something remarkable happens. To predict text well, the model has to pick up grammar, facts, writing styles, and even some reasoning patterns. The result is a base model: great at continuing text, but not yet a good conversation partner. Ask it a question and it might just continue with more questions.

2. Fine-tuning: learning to be an assistant

Next, the model is trained further on a smaller set of example conversations, where a prompt is paired with a high-quality, helpful reply written or checked by people. This teaches it the shape of a good answer: follow the instructions, stay on topic, be clear.

3. Learning from human feedback

Finally, people compare several answers from the model and mark which ones are better: more helpful, more accurate, safer. The model is then trained to produce more answers like the preferred ones. This approach is often called RLHF (reinforcement learning from human feedback). Companies combine it with other techniques and safety rules, but the goal is the same: turn a raw text predictor into a useful, polite assistant.

The context window: ChatGPT’s short-term memory

When you chat, ChatGPT doesn’t remember you the way a friend does. Instead, each time it replies, it re-reads the conversation so far: your messages, its earlier answers, and any hidden instructions. All of that has to fit inside its context window, the maximum number of tokens it can look at in one go.

Context windows have grown a lot, but they are still a limit. In a very long chat, the earliest parts may drop out of view or get less attention, which is why a model can “forget” something you said much earlier. Some apps add separate memory features on top, but the context window is what the model itself can see.

Why ChatGPT sometimes makes things up

Remember: the model is trained to produce text that sounds likely. Most of the time, likely and true line up. But when it lacks the right information, it can still produce something that sounds perfectly plausible: a made-up quote, a book that doesn’t exist, or a wrong date. This is called a hallucination.

So what is ChatGPT good at?

Because it has absorbed so many patterns of language, an LLM is excellent at drafting, rewriting, summarising, translating, brainstorming, explaining tricky ideas in simpler words, and helping with code. The quality of what you get depends a lot on what you ask. Clear, specific instructions lead to better predictions, and our guide to prompt engineering for beginners shows you how to write them.

LLMs are also becoming the “brains” of tools that can take actions, not just chat, like searching the web, filling in spreadsheets or booking things. If you’re curious about that next step, read what an AI agent is and how it works. And if you have wondered how AI turns words into pictures, a related but different technology is explained in how AI image generators work.

Frequently asked questions

That’s the whole trick: tokens in, one predicted token out, repeated at lightning speed, built on patterns learned from an enormous amount of text. Next time ChatGPT answers you, you will know exactly what is happening behind the blinking cursor.