Models and prompts

What a model is

A large language model, or LLM, is a program trained on huge amounts of text to predict what text should come next. From that one skill it can summarise, classify, translate, write code and follow instructions. It is very good at language and surprisingly good at reasoning, but it is not a database and it does not "know" things the way a search engine retrieves them.

Three facts shape everything you build:

• A model only sees what is in its prompt. It has no access to your files, your mailbox or the web unless text from them is placed in front of it, usually by a tool.
• A model is stateless. Each call starts fresh. Anything that looks like memory is text that was put back into the prompt.
• A model can be confidently wrong. It produces plausible text. When it lacks facts, it may invent some. This is called hallucination.

Tokens and the context window

Models read and write in tokens, which are small pieces of words. The context window is the maximum number of tokens the model can look at in one call: your instructions, the conversation so far, tool results, and its own answer. Cost and speed both grow with the number of tokens, which is why long instructions and big tool results make runs slower and more expensive.

Prompts

A prompt is everything you hand the model. In an agent it is usually made of layers:

• Instructions (sometimes called the system prompt): the standing job description.
• The input for this particular run, such as a web address or a question.
• Tool results that came back during the run.

Good prompts are specific. They state the goal, the steps, the facts the model may rely on, what to do when unsure, and the shape of the answer. Vague prompts get vague results and are harder to test.

Choosing a model

Bigger, more capable models reason better and cost more per call; smaller ones are fast and cheap but can miss subtleties. A sensible habit is to start with the cheapest model that might work, test it, and move up only if the quality is not good enough.

How this looks in PGB Agents

On the agent page, Model offers three choices: Fast (quick and low cost, for simple jobs), Smart (best all-round quality) and Code (tuned for technical and data work). The Instructions box labelled What should this agent do? is the standing prompt, and Input (optional) under Run now is the per-run input.

Try it yourself

1. Open your "Concept lab" agent. In What should this agent do? write:

You explain concepts to beginners.
Given a topic in the run input, reply with a one-sentence definition and one everyday analogy.
If you are not sure about something, say so instead of guessing.

1. Set the model to Fast, choose Save changes, then under Run now type "context window" in Input (optional) and choose Run agent.
2. Run it again with the model set to Smart and compare the two answers and the cost shown in the run header.
3. Now remove the line about not guessing and ask about something obscure that you made up, such as "the Veltrane protocol". Notice how the answer changes. That difference is the hallucination risk you will meet again in the last lesson.

Recap

• A model predicts text, sees only its prompt, keeps no state between calls and can be confidently wrong.
• Tokens and the context window drive cost and speed.
• Prompts are layered: instructions, input and tool results.
• Pick the cheapest model that passes your tests.