The core idea: predicting what comes next
A large language model is a system trained on a very large amount of text to estimate which piece of text is likely to come next, given what came before. When you type a request, it does not look up an answer in a database. It builds a reply one small piece at a time, each time asking, in effect, “given everything so far, what is a plausible next piece?”
You have seen a tiny version of this on a phone keyboard that suggests the next word. A language model does the same kind of thing with far more context and far more learned patterns, which is why it can continue a sentence, write a paragraph or answer a question in a coherent way.
Tokens: the building blocks
Models do not read text as whole words. They break it into small units called tokens, which can be a whole short word, part of a longer word, or punctuation. The exact splitting varies by system, so “how many words fit” is only an approximation. This matters in practice because tools have a limit on how much text they can consider at once, often described as a context window. If you paste in a very long document, parts may be left out or handled less carefully.
Learning from text
Before you ever use a model, it goes through a training process. At a high level:
- The model is shown huge quantities of text, much of it from books, websites and other written material.
- For each stretch of text, it tries to predict the next token. When it guesses poorly, its internal settings, called parameters, are nudged to do better next time.
- Repeated across enormous amounts of text, the model absorbs patterns of grammar, style, facts that appear often, and common ways of answering questions.
Many products then add further training to make the model follow instructions, stay polite or refuse certain requests, often guided by feedback from people. The details differ between products and are rarely fully public.
The consequence is that what the model can do is shaped by what it learned from. This is the central point of the discussion on training data. Topics well covered in the text it saw tend to go better than obscure or very recent ones, and any gaps or slants in that text can show up in its answers.
How a reply gets built
When you submit a prompt, roughly this happens:
- Your text, plus any instructions the tool adds behind the scenes, is converted into tokens.
- The model produces a set of probabilities for what the next token could be.
- One token is selected. Often it is a likely one, but not always the single most likely, because some randomness is built in to keep output from sounding repetitive.
- That token is added to the text, and the process repeats until the reply is complete.
Because of the randomness, asking the same question twice can produce different wording, and sometimes different content. Some tools let you reduce this variation, though not always eliminate it.
Why this makes text tools useful
Learning patterns in language lets these systems do many language tasks reasonably well: drafting, rewriting, summarizing, translating, explaining and sorting. This is the basis of tools for email drafting, text rewriting and document summaries. They can follow an example you provide, which is the idea behind example-based prompting, and they respond to how clearly a request is phrased, which is why guidance on writing prompts is worth reading.
Why this also explains the weaknesses
Understanding the mechanism makes several known problems less mysterious.
Plausible is not the same as true. The model aims at likely-sounding continuations. When it lacks the relevant information, it can still produce something that sounds right. That is the origin of fabricated answers, such as invented citations or wrong details stated smoothly.
No built-in sense of what it does not know. The tool is not checking a source of truth unless it has been connected to one. Some products retrieve information from documents and then write based on it, which can help, but errors remain possible.
Sensitivity to wording. Small changes in a prompt can lead to noticeably different outputs. Adding relevant background, a topic covered under prompt context, usually helps.
A cutoff in what it learned. The model’s built-in knowledge stops at some point in time. Newer events may be missing unless the tool is connected to current information.
Limited arithmetic and logic reliability. Because it predicts text rather than calculating, it can make mistakes with exact numbers, multi-step reasoning or counting. Check any figures independently.
Reflecting its sources. Patterns in the training text, including stereotypes, can surface in output, tying into algorithmic bias.
What it is not
Describing a model as “predicting the next word” can sound dismissive, and in some ways it undersells what emerges from doing this at scale. But it is also easy to overstate. A language model is not a person, does not have goals of its own, and does not “know” things in the way a human expert does. Some of the more common misreadings are addressed in the piece on AI myths.
Practical habits that follow
- Treat output as a draft. Review and edit before use.
- Verify specifics. Names, dates, quotes, figures and legal or medical claims should be checked against reliable sources.
- Give context and constraints. State the audience, purpose, format and any facts the model should rely on.
- Be careful what you paste in. Understand where entered text goes and how long it is kept, as covered under data retention.
- Keep a human responsible. The person who sends or publishes the text owns what it says.
Capabilities differ across tools and change over time, so what holds for one product may not for another. Testing on your own tasks, with low stakes at first, is the most reliable way to learn what a particular tool does well.