LLM Foundations

A large language model is a neural network trained on massive amounts of text to predict the next token in a sequence. Through that single training objective — given everything before this point, what comes next? — the model learns language structure, factual knowledge, reasoning patterns, and the ability to follow instructions.

That description is deceptively simple. The result is a system that can generate code, translate languages, summarize documents, answer questions, and produce structured data on demand. Understanding how it works, what it can do, and where it fails gives you the foundation to use it well.

What an LLM actually is

Strip away the marketing and an LLM is two things:

Model weights — billions of numerical parameters learned during training. These encode everything the model "knows."
Architecture code — the neural network structure (a transformer) that runs those weights to produce output.

The model operates on tokens, not raw text. A tokenizer splits input into sub-word units from a fixed vocabulary. Each token maps to a number, the model operates on numbers, and output numbers decode back to text. A rough rule: one token is about three-quarters of an English word.

Tokens are the unit of everything: pricing, context window size, latency. Longer input costs more and leaves less room for the response.

The transformer architecture

Before 2017, language models used recurrent neural networks that processed text one token at a time, left to right. They were slow (no parallelization) and forgot the beginning of long sequences by the time they reached the end.

The transformer (Vaswani et al., 2017) solved both problems. Instead of processing tokens sequentially, it processes the entire input in parallel using a mechanism called self-attention. Every token can attend to every other token regardless of distance, so the model captures long-range relationships in one pass.

Self-attention works by computing, for each token, how relevant every other token is to it. The model learns which relationships matter — "capital" attending strongly to "France" when predicting "Paris" — through training. Multiple attention heads run in parallel, each learning different types of relationships.

How LLMs are trained

Training happens in stages, each building on the last.

Pretraining. The model reads enormous text corpora (books, articles, code, web pages) and learns to predict the next token. This requires thousands of GPUs running for weeks or months. The result is a base model — a text-completion engine that generates plausible continuations but does not reliably follow instructions.

Post-training (alignment). The base model is transformed into a useful assistant through two steps. Supervised fine-tuning (SFT) trains it on curated examples of ideal assistant behavior. Then reinforcement learning from human feedback (RLHF) uses human preference data to further tune the model toward responses people prefer — learning subtle quality distinctions like conciseness versus thoroughness.

Reasoning training. More recently, reinforcement learning teaches models to reason step by step before answering. The model generates a chain of thought — breaking problems into sub-steps, exploring alternatives, verifying work. This produces reasoning models (like OpenAI's o-series) that are better at math, logic, and complex multi-step problems, at the cost of higher latency.

How inference works

When you send a request, the model generates its response one token at a time (autoregressive generation):

Your full input is tokenized and processed.
The model produces a probability distribution over its vocabulary — predicting the most likely next token.
A token is selected (influenced by temperature and sampling parameters).
That token is appended and the process repeats until a stop condition.

Everything must fit in the context window — your prompt, conversation history, any injected context, and the generated response. Modern models offer windows from 128K to over 1M tokens, but the budget is always finite.

What LLMs are good at

Reasoning and analysis — breaking down problems, comparing options, explaining concepts
Content generation — writing prose, email, reports, and code
Summarization — distilling long documents into key points
Translation — between natural languages or between formats (JSON to prose)
Code generation — writing, explaining, and debugging code
Classification and extraction — categorizing text, pulling structured data from unstructured input
Structured output — generating JSON, YAML, or other machine-readable formats

What LLMs struggle with

No real-time knowledge. Training data has a cutoff. The model does not know about events after training unless you supply them.
Hallucination. Models generate confident but incorrect statements. They predict plausible text, not verified facts.
No persistent memory. Each API call is stateless. The model remembers nothing between calls unless you include prior context.
Limited precision. Math errors, logic mistakes, and counting failures still occur, especially without chain-of-thought reasoning.
Non-determinism. The same input can produce different output across calls.
No ability to act. An LLM generates text. It cannot send email, query databases, or call APIs without external tooling.

Where LLMs fit in a system

An LLM alone is a text-in, text-out function. To build useful applications, you layer on top: system prompts for behavior, tools for action and data retrieval, memory for conversation state, guardrails for safety. These layers turn a language model into an agent or an application component.

The model is the reasoning engine. Everything else — what it knows, what it can do, how it behaves — is determined by what you put around it.