LLM Cheat Sheet

I saw a recent post on X that called out a well known software engineer for not knowing how LLM’s actually work. And I realised, you know what? I’ve been using LLM’s for over a year now and I don’t really know how they work either!

So I did some reading into it and found that while most things I had picked up or intuited, there were definitely some useful learnings that really help understand LLM’s, their limitations, what they’re good at, and how to use them better.

I’ve distilled that knowledge into this Cheat Sheet, and I have ensured it is useful for any audience, not just software engineers.

Definitions

Let’s start with some definitions and key concepts as these are foundational.

Large language model (LLM). A type of AI system trained to predict text. Given some text, it produces the most fitting continuation, one chunk at a time. Everything an LLM does — answering questions, writing code, summarizing documents — is a form of that one operation. Claude, ChatGPT, and Gemini are all LLMs.

Token. The unit an LLM reads and writes. A token is roughly a word-piece: common words are usually a single token, while rarer words split into several. The model never sees letters or words exactly as you do — it sees sequences of tokens. This matters in practice, because length limits and usage costs are all measured in tokens, not words, and unusual text (code, other languages, long names) fragments into more tokens than you’d expect.

Parameters (or weights). The billions of numbers, learned during training, that encode everything the model “knows.” They are fixed once training ends. Talking to a model does not change its weights.

Training. The process that produces those parameters. The model is shown vast amounts of text and repeatedly tries to predict the next token; each wrong guess nudges its parameters toward a better one. Done across trillions of tokens, this is how the model absorbs grammar, facts, and reasoning patterns. Training happens once, before you ever use the model, and ends at a fixed cutoff date.

Context window. The maximum amount of text the model can take in at one time. Your instructions, the conversation history, any files the model has opened, and the response it’s generating all share this single budget. If something isn’t in the window, the model has no access to it.

Attention. The mechanism the model uses to decide what matters. As it reads, it weighs every token against every other token to work out which ones are relevant to what it’s generating. Attention is what lets the model connect a pronoun to the right noun or a question to the relevant detail — and because that weighing is spread across everything in the window, a fuller window means thinner, less reliable attention.

Inference. The act of running a trained model to produce output — what happens every time you send a message. It is distinct from training, which happened once, beforehand.

Harness. The software wrapped around the model that turns a raw text predictor into a usable tool. The model itself only reads a window and emits tokens; the harness is everything else — it assembles what goes into the window, gives the model tools it can call (search, running code, reading files), executes those calls, and feeds the results back in. A chat app and a coding assistant use the same kind of underlying model; the difference in what they can do is largely a difference in harness.

Key concepts

Learning happens once, before you ever use it

An LLM is built in a massive training phase. The model is shown enormous quantities of text and plays one relentless game: predict the next token. Each time it guesses wrong, its parameters are nudged slightly toward a better guess. Repeated across trillions of tokens, this process forces the model to internalize grammar, facts, reasoning patterns, and styles — not because anyone programmed those rules, but because predicting text accurately requires picking them up.

A second stage shapes the model’s behavior: it’s tuned, with human and automated feedback, to be helpful, to follow instructions, and to decline harmful requests. This changes how the model acts, not what it fundamentally knows. A well-mannered model can still be confidently wrong, because its knowledge all came from that first phase.

The practical consequence: a model’s knowledge is frozen at its training cutoff date. It has no built-in awareness of anything that happened afterward — recent events, new software versions, today’s prices — unless it can reach out through a tool such as web search.

Inference is pattern completion over a window

When you send a message, the model reads everything currently in its context window and produces the most plausible continuation, token by token. Two things follow from this, and they explain most of what people find puzzling.

First, the model has no memory between turns. It re-reads the entire window every single time it responds. The “memory” you experience in a conversation is simply the transcript being present and read again. The reliable way to give the model something is to make sure it’s in the window — in your prompt, in a file the model opens, or in a persistent project file that loads automatically at the start of a session.

Second, attention is shared across everything in the window. The model weighs every token against every other token to decide what matters — this mechanism, called attention, is the heart of how modern LLMs work. But that weight is finite and spread across the whole window. As the window fills up, earlier instructions carry less force, and quality tends to soften well before the window is technically full. A focused window keeps the model sharp; a cluttered one dilutes it.

Fluency is not truth

Because the model generates the most plausible continuation, its output is smooth and confident regardless of whether it’s correct. Fluency reflects how well text fits learned patterns; truth is a separate property those patterns don’t guarantee. When a model produces something that sounds right but isn’t, it’s often called a hallucination — and it’s a feature of the mechanism, not a bug waiting to be patched. The tone of an answer tells you nothing about its accuracy.

Thinking is just more tokens

A model can only generate tokens, so giving it room to generate intermediate working-out before the final answer measurably improves results on hard, multi-step problems. Those reasoning tokens become part of what the model reads as it continues, letting it build toward an answer instead of committing to one immediately. This helps on genuinely structured problems — complex logic, tricky debugging, careful analysis — and adds little to simple, single-step tasks.

Extending what the model can do

A bare model only reads a window and emits tokens. Everything beyond that — reaching live data, following your team’s process, taking real actions — comes from the harness around it. Skills and MCPs are the next thing you need to know about.

MCP (Model Context Protocol) is about access. It’s an open standard for connecting a model to external systems — files, databases, APIs, a code host, a chat tool. An MCP server exposes a set of tools the model can call, and the harness handles the calls and feeds results back into the window. If a task needs a live connection to something outside the model, MCP is the layer that provides it.

Skills are about knowledge — specifically, procedural know-how. A skill is a small folder containing a set of instructions that teaches the model how to do a particular task well: your code-review standards, a document-formatting routine, a deployment checklist. The useful design detail is how they load. Only a short description sits in the window by default; the full instructions are pulled in only when the task actually calls for them. This progressive disclosure keeps the window focused — which, given everything above about attention, is exactly the point. A skill is best thought of as the onboarding guide you’d hand a new hire, written once and reused.

Guidelines

Everything above reduces to one idea: you are working with a pattern-completer that has a finite, attention-limited working memory and no guarantee of truth. These guidelines all follow from that.

Put what matters into the window. Don’t rely on the model to remember or infer context it can’t see. State your constraints, paste the relevant material, and point the model at the right files. Anything important should be present and easy to find.

Keep the window focused. On the other hand, only include what is relevant. Irrelevant material dilutes attention and accelerates the drop in quality. When you start a genuinely new task, start a fresh session or chat so the model gets a clean, well-weighted space to work in.

Verify the important parts. Treat confident output as a strong draft, not a final answer. Leverage tests, checkpoints, known correct examples as a way to tell the model if the output is correct or not. Especially anything that requires arithmetic calculations — check the transcript to see that it didn’t just guessthe answer, it actually called a tool / command / script to do the calculation.

Ask the model to look things up when recency matters. For current events, the latest versions, or anything time-sensitive, prompt the model to search or use a tool. Given real sources, it answers from solid ground instead of reconstructing from frozen memory.

Plan before building on anything large. For multi-file or unfamiliar work, having the model map its approach first costs a few minutes and saves the much larger cost of confidently building the wrong thing. (Many coding assistants offer a dedicated plan mode for exactly this.) Work you can describe in a single sentence is fine to do directly.

Match effort to the task. Let the model reason at length on hard, multi-step problems where the working-out earns its keep. Keep things light and fast for trivial requests, where extra deliberation adds nothing.

Course-correct early. Because each turn builds on the window, catching a wrong direction quickly keeps the context clean and the reasoning on track. When a path has truly gone bad, a clean reset with a sharper prompt — one that folds in what you just learned — beats piling correction on top of correction.

Give the model something to check itself against

The deepest version of verification isn’t something you do after the fact — it’s something you hand the model up front, so it can correct itself.

Recall the core problem: the model produces fluent output with no built-in sense of whether it’s right. Left on its own, it has nothing to push against; one plausible-but-wrong step just leads to the next. What changes everything is giving it an external oracle — an objective check it can run and read the result of. Because that result lands back in the window, the model can see its own failure and revise, turning a one-shot guess into a loop that converges on something correct.

The two most useful oracles are tests and known-good examples.

Tests. A failing test is unambiguous in a way prose never is. When the model can run a test suite, it gets a hard signal — pass or fail, with the exact error — that it can iterate against without you in the loop. This is why describing the desired behavior as tests, or asking the model to write tests before the implementation, is so effective: you’ve handed it a ground truth to aim at and a way to know when it’s hit it. The same logic extends to anything with a checkable result — a script that must run, output that must match a schema, a build that must succeed.

Known-good examples. When there’s no test to run, a worked example does similar work. A sample input paired with its correct output, a reference implementation, or a “here’s what good looks like” snippet gives the model a concrete target to match rather than a vague instruction to interpret. Examples pin down intent far more reliably than description alone, and they let the model compare its work against a standard instead of inventing one.

The shared principle: don’t rely on the model to judge its own correctness from the inside. Give it an outside reference — a test that fails loudly, an example it can measure against — and you convert a confident guesser into a system that can find and fix its own mistakes.

Choice of model

Before starting some new work, do some research via Google or even ask a model itself about what model is best suited to your particular task.

Each model will have a suite of benchmarks available to show where it excels and where it lacks compared to other models.

The most powerful “frontier” models from Claude and OpenAI are fairly well rounded, but depending on your budget they can quickly use up their quotas / limits.

Further learning

I would actually recommend to just ask your LLM further questions for anything that you don’t understand or that you want to dive deeper into. That’s mostly how I wrote this post in the first place. Open to feedback on this one! Thanks for reading.