What an LLM actually is

A large language model is a function with one job: given some text, predict the next token (a word or a piece of a word). It returns a probability for every token in its vocabulary. The program that calls it picks one, appends it to the text and asks again. Repeat that a few hundred times and you get an essay, a SQL query or a tool call. Everything in this course (RAG, agents, MCP, observability) is engineering around that loop.

1. The generation loop

Three consequences follow directly from this picture, and they explain most LLM behaviour you will see:

2. Build a tiny language model

The smallest possible language model counts which word follows which (a bigram model). Real LLMs are far more capable, but the interface is identical: context in, next-token probabilities out. Here it is, trained on the text of this site's SQL course:

import random
import re
from collections import Counter, defaultdict

from site_docs import load_sections          # the lessons of this website as plain text

text = " ".join(d["text"] for d in load_sections(courses=["sql"])).lower()
words = re.findall(r"[a-z_']+|[.,]", text)

follows = defaultdict(Counter)              # word -> Counter of the words that came next
for current, nxt in zip(words, words[1:]):
    follows[current][nxt] += 1

def next_word_probs(word, k=4):
    counts = follows[word]
    total = sum(counts.values())
    return [(w, round(n / total, 2)) for w, n in counts.most_common(k)]

print(f"trained on {len(words):,} words, vocabulary {len(follows):,}")
print("after 'primary':", next_word_probs("primary"))
print("after 'group':  ", next_word_probs("group"))
print("after 'left':   ", next_word_probs("left"))

random.seed(3)
def generate(word, n=14):
    out = [word]
    for _ in range(n):
        counts = follows[out[-1]]
        out.append(random.choices(list(counts), weights=list(counts.values()))[0])
    return " ".join(out)

for start in ["the", "an", "indexes"]:
    print(">", generate(start))

The probabilities are real statistics: "primary" is almost always followed by "key", and "group" by "by". Every adjacent pair in the generated text really occurs in the course. Yet the sentences as a whole say nothing true, and some say nothing at all. That is hallucination in miniature. The model continues text plausibly and has no notion of whether the result is correct. A real LLM conditions on thousands of previous tokens instead of one, which is why its output is coherent over pages. Coherent is still not the same as verified.

3. How a model is made: three training stages

4. Hallucination: why, and what you do about it

A hallucination is fluent output that is false or unsupported: an invented function name, a citation to a paper that does not exist, a confident wrong number. It is not a bug that will be patched away. It follows from the objective. The model is trained to produce likely text, and when it lacks the facts, a likely-sounding answer is still likely text.

5. What LLMs are good and bad at

6. Where models come from

The skills in this course do not depend on the vendor. Retrieval, tool use, MCP, tracing and evaluation work the same way whichever model sits in the middle. The Python examples use the OpenAI SDK because most providers and local servers (Ollama and vLLM included) offer an OpenAI-compatible endpoint. Change base_url and the code keeps working.

Recap