Skip to main content

Prompt your agent for grounded answers

When your agent answers questions from your knowledge, or makes decisions based on it, two behaviors matter most: expressing uncertainty when the knowledge does not cover the question, rather than making something up, and citing sources so users can verify claims. Kapa's Prebuilt Agents produce both out of the box, but nothing about them is automatic when you build your own agent: you need to explicitly instruct it.

If you are using the Kapa Agent SDK, these behaviors are already built in. If you are building your own agent with the API or MCP, you will need to add this guidance yourself, in both the tool description and the system prompt.

It helps to be precise about the mechanics. Your agent is an LLM running in a loop, and Kapa retrieval enters that loop as a tool: the model sees the tool's name and description alongside every other tool it has, decides when to call it, and receives the returned chunks in its context to reason over. That gives you exactly two places to shape the behavior, and the split applies to any tool: the tool description tells the model what the tool does and what to expect back from it, while the system prompt guides how to use the tool and how to handle its results. Here we show what this looks like for Kapa retrieval specifically.

Tool description​

The tool name and description are where you set baseline expectations about what the tool does and what it returns. The name matters because it is the first signal the model uses when deciding which tool to call. If you are using a Kapa hosted MCP server, both are already configured for you, and you can customize them; see Customize the MCP tools for when that helps. The default tool name is search_<PRODUCT_NAME>_knowledge_sources and the default description is:

Perform semantic retrieval over the documentation and other knowledge
sources of [Product Name] and return the most relevant chunks for a
given query. A "chunk" is a short, self-contained snippet of text taken
from a single page or item within these sources (for example, part of a
documentation page) and includes its source URL and markdown content.
Chunks are returned in descending order of relevance to the query. If
the knowledge sources do not contain information relevant to the query,
the returned chunks may be only weakly related or entirely unrelated.
Use this tool anytime you need information about [Product Name].

Notice that the description already enforces some of the behavior we want. It tells the model that results include source URLs and content, and critically, that chunks may be weakly related or entirely unrelated to the query. This primes the model to treat results with appropriate skepticism rather than assuming everything returned is relevant.

If you are building your own tool with the HTTP API, use a similar description. The important elements are: explain what the tool returns (chunks with source URLs and content), note that results may not always be relevant, and tell the model when to use it.

System prompt​

The system prompt is where you get precise about how the agent should use the tool and handle its results. Below is what we think works well for handling uncertainty and citations. You can use it as a starting point or do your own thing entirely. Keep in mind that different LLMs will behave differently given the same set of instructions:

You are an AI assistant for [Product Name], [product description].

You have different tools to interact with the platform on behalf of the
user. You also have access to a knowledge base search tool to answer the
user's questions and fill in your own understanding of the product.

...your other instructions...

## Knowledge Base Search

Use the search_knowledge_base tool whenever you need information about
[Product Name], whether to answer a user's question directly, to
understand parts of the product in connection with the user's task, or
to interpret results and inputs from other tools. Never rely on your
inherent knowledge or any other tool for product-specific information.

When using the results, follow these rules:

1. Review the content of each knowledge source document carefully and
   assess its relevance to the user query before using it in your answer.
   Some documents may appear relevant at first but are not. If you do not
   find enough information in the knowledge sources to answer the query,
   clearly state that the knowledge sources do not contain enough
   information. Never try to make up an answer.
2. Answer the user query solely based on the relevant knowledge sources.
   Explicitly state your uncertainty with phrases like "I'm sorry, but
   there's no information about", "The knowledge sources do not explicitly
   mention" or similar, if you cannot find enough information in the
   knowledge sources to provide a confident answer. State your uncertainty
   at the beginning of your answer — the user must be made aware of any
   limitations upfront.
3. Cite documents that include a source URL. Format each citation as
   [[short title](URL)] using a 1-4 word title. Place the citation
   immediately after the sentence it supports. If two or more documents
   support the same sentence, include at most two citations inside the
   same brackets separated by a semicolon.