What Is a Reasoning Model?

A reasoning model is a large language model trained to work through a problem step by step before it gives an answer. It generates an internal chain of intermediate thoughts, checks and corrects itself, and only then writes the reply. The extra thinking costs tokens and time, and in return it gives noticeably better results on maths, code, planning and analysis that needs several steps.

How Reasoning Models Work

Early on, people got better answers by asking a model to “think step by step” in the prompt. Reasoning models build that behaviour into the model itself through training.

The core technique is reinforcement learning on problems with checkable answers: maths with a known result, code that must pass tests, puzzles with one solution. The model tries, gets rewarded when the final answer is right, and gradually learns to produce longer, more careful lines of thought. The DeepSeek-R1 paper showed publicly that this kind of reasoning can be incentivised through reinforcement learning without human-written reasoning examples, and that the resulting patterns can be used to improve smaller models.

At run time, the model writes thinking tokens first and the visible answer second. Vendors expose controls for how much it thinks. In Anthropic’s documentation, for example, thinking can be given an explicit token budget or left to the model to size per request, and the thinking tokens are billed as output tokens. Other vendors offer similar “effort” settings.

When to Use a Reasoning Model

Reasoning helps when the task has several dependent steps and a wrong early step spoils the result:

  • Debugging or refactoring code across several files.
  • Planning a sequence of tool calls for an AI agent.
  • Reconciling conflicting documents or contract clauses.
  • Writing a complex SQL query from a business question (see text-to-SQL).
  • Checking a draft against a long list of rules.

It does not help, and only adds cost and delay, when the task is a single step: classifying a ticket, extracting a field, rewriting a sentence or answering from one retrieved paragraph. A fast model or a small language model is the better tool there.

Common Failure Modes

  • Latency. A reasoning call can take many seconds. That is fine for a background job and wrong for a voice agent or live chat.
  • Cost surprises. Thinking tokens are usually invisible in the final answer but still billed. Without limits and monitoring, a busy endpoint can cost several times what was planned.
  • Overthinking. On easy inputs the model may reason at length and talk itself into a worse answer.
  • Trusting the visible reasoning. The summary of the model’s thinking is useful for debugging, but it is not a guaranteed account of how the answer was produced. Judge the answer, not the explanation.
  • Still not a calculator. Better reasoning reduces errors in arithmetic. It does not remove them.

Reasoning models are often the “planner” in agent orchestration, while cheaper models do the routine steps. Model routing decides per request whether reasoning is worth paying for. Teams compare reasoning and non-reasoning models on the same tasks through LLM evaluation.

In Practice

Reasoning is switched on step by step, where an evaluation shows it changes the result, rather than for a whole system. A planning step or a hard classification may justify a higher effort level, while extraction and formatting run faster without it. Thinking tokens are tracked per step in tools such as LangSmith, since they can multiply the cost of a call, and each step has a budget cap so one hard case cannot run on indefinitely.

We use reasoning models where they pay for themselves in AI agent development and AI workflow automation projects.

Back to Glossary

Let's talk about your project

We'll help you harness AI and ML technologies to create innovative software solutions

File requirements: pdf, doc, docx, rtf, ppt, pptx — max 15 MB

No file selected