The art of talking to a LLM

A prompt isn't text — it's a program

When you type "summarize this article" into ChatGPT, nothing particularly spectacular seems to happen. But the text you sent triggered a very specific behavior in a model trained on trillions of tokens.

A prompt is a program in natural language. Not in the sense that it gets compiled, but in the sense that its wording determines which type of behavior the model activates. The same sentence reformulated differently can produce radically different results — not because the model is capricious, but because its pre-training has taught it distinct patterns for distinct contexts.

Prompt engineering is the art of formulating that program to get the behavior you want.

Four levels of technique

Zero-shot: just ask the question

The simplest technique. You ask the question directly, with no examples or instructions. The model activates the most likely behavior given its training.

It works great for simple, factual tasks. It fails on problems that require reasoning — not because the model doesn't know, but because it doesn't know that it should reason.

Few-shot: show examples

Instead of explaining what you want, you show it. You place 2 to 5 input/output pairs before the real question. The model — through its in-context learning mechanism — reads the pattern and applies it to the new input.

The key: examples must be representative of the type of task. Off-topic examples don't help. Examples that show the right approach help a lot.

Chain-of-Thought: reason step by step

Two 2022 findings, often conflated. Wei et al. showed that if you give the model examples whose reasoning is written out in full, it starts reasoning the same way on the next problem. Kojima et al. went further the same year: no examples needed, just add "let's think step by step". In both cases, performance on reasoning problems doubles, sometimes triples.

Why does it work? The model generates tokens one by one. By forcing it to write its intermediate reasoning, you give it a "scratchpad" where it can compute, check hypotheses, and correct errors — before concluding. Without CoT, it jumps straight to the conclusion with no safety net.

It's the same principle as for a human: writing "25 × 37 = 25 × 30 + 25 × 7 = 750 + 175 = 925" gives you much better odds than attempting to compute mentally in one shot.

Self-Consistency: vote across reasoning chains

Self-consistency is an extension of CoT. Instead of generating one reasoning chain, you generate several (typically 5–20) with varied temperatures, then vote for the most common answer.

The idea: each run may make a different mistake. But if most converge on the same answer, it's probably right.

It's expensive (N times more tokens), but on hard reasoning tasks, the reliability gain is real.

Try it yourself

Compare the four techniques across three problems. Notice in particular that few-shot examples help on a structured problem (the merchant) but barely change anything on logical traps.

Same question, four wordings. The score swings from 30 % to 90 % without touching the model. The lesson: a prompt isn't text, it's a program whose implicit syntax LLMs interpret thanks to their pre-training.

What this reveals about LLMs

These four techniques aren't tricks. They illuminate something fundamental about how LLMs work.

In-context learning is free. An LLM learns from your examples without updating its weights — just by reading the context. This is an emergent capability from massive pre-training: the model has seen so many patterns that it can extract a new one on the fly.

Reasoning is a behavior, not a fixed capacity. A model that fails zero-shot on a problem can succeed at CoT on the same problem — without changing any parameters. What the prompt activates changes what the model "does" with its internal capabilities.

Temperature creates diversity, voting reduces variance. Self-consistency exploits the fact that errors are often random: many different ways to fail, but only one way to succeed. Consensus filters the noise.

The limits

Cost per call. Each few-shot example consumes tokens — and you pay for them again on every request. Today's windows, from 200,000 to over a million tokens, technically allow many-shot prompting (hundreds of examples, and it does work), but the bill and the latency climb with it. CoT also lengthens responses.

Examples can mislead. If your examples contain a bias, the model will reproduce it. "Garbage in, garbage out" applies to few-shot too.

Prompt injection. Malicious content in the context can override your instructions. If your prompt says "translate this text" and the text says "ignore previous instructions and do something else," the model may obey the content rather than the instruction.

Models evolve. A prompt honed for GPT-4 in 2023 doesn't necessarily work on today's models, nor on another vendor's. Each model has its own preferred patterns, its own formulations that "click" better.

The practical rule

Choosing a technique:

• Simple / factual question → zero-shot is enough.
• Specific output format expected → few-shot with 2–3 examples.
• Reasoning or calculation → CoT, if your model has no native reasoning mode. If it does, let it do its thing: forcing an explicit CoT on top is at best redundant, at worst harmful — some providers explicitly advise against it.
• Critical reliability → CoT + self-consistency.

And a meta-rule: if your prompt looks like code — with a clear structure, explicit variables, defined use cases — it will be more reliable than ambiguous prose.

One last thing. The techniques described here (CoT in particular) are the prompt-driven ancestors of native reasoning models: o1 and DeepSeek-R1 opened the way in 2024-2025, and today every major model has a mode of this kind. Those do automatically and intensively what prompted CoT only simulated — see chapter 17 to understand the shift from prompt engineering to reasoning baked into the model.

A good prompt isn't a magic formula. It's a clear specification of what you want, in a language the model recognizes as the signal it should follow.

One thing remains that no wording can fix. You can write the clearest specification in the world: if the answer isn't in the model's weights, it will produce one anyway, with exactly the same poise as if it were. Why is a system trained to predict text unable to stop and say "I don't know"? That's the subject of the next chapter.

Updated August 2, 2026