---
title: "Optimize cost and performance | Grafana Cloud documentation"
description: "Use Agent Observability data to reduce LLM costs, improve cache efficiency, and tune agent performance."
---

> For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt).

# Optimize cost and performance

Agent Observability captures detailed token usage, cost, and cache data for every generation. Use this data to find optimization opportunities and reduce your AI spending.

## Before you begin

- Configure your agent to export generations and OpenTelemetry metrics to Grafana Cloud. For instructions, refer to [Get started with Grafana Cloud](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/get-started/grafana-cloud).
- Collect representative traffic for the agents and models you want to compare.

## Analyze token usage

Open **Analytics** and review the tokens and cost dashboard. Look for:

- **High-token agents**: agents that consistently use more tokens than expected may have verbose prompts or unnecessary context.
- **Model selection**: some calls may use expensive models where a cheaper alternative would produce equivalent results.
- **Reasoning tokens**: if reasoning tokens are high, check whether thinking mode is enabled unnecessarily.

For recorded generation costs and observed token and request prices, refer to [Use Prometheus metrics directly](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/observe-and-troubleshoot/dashboards/#use-prometheus-metrics-directly). These metrics let you build custom cost queries; multiplying aggregate usage by current prices can differ from recorded costs.

## Improve cache efficiency

Prompt caching reduces costs by reusing previously processed prompt prefixes. Agent Observability tracks cache read and cache write tokens for each generation.

To improve cache rates:

- Keep the system prompt stable. Changing the system prompt frequently invalidates the cache.
- Place static content (system prompt, tool definitions) at the beginning of the message sequence.
- Minimize dynamic content that changes between requests.

Open **Usage and cost** and select **Cache hit rate** to track cache reuse over time. Compare agents or models with **Break down**, then inspect the **Cache hit rate** column in the conversations table. Select **Token type** to compare cache read and cache write counts instead of a percentage.

### Monitor cache efficiency across agents

When prompt cache analysis is enabled, use **Agents** to identify which agents benefit from caching and which lose spend to uncached tokens:

- **Agents table**: inspect the **Cache** column for each agent’s hit rate, or **No activity** when no model reads from or writes to the cache. Agents with no input tokens in the selected window show a dash (`—`) and sort last. The **Uncached / day** column estimates the extra cost incurred from input tokens that weren’t read from cache.
- **Top uncached cost**: review the five agent and model pairs losing the most to uncached input. Each row displays the agent name, model name, uncached cost per day, and hit rate. If a model has a prompt price but no cache-read price, it displays **No cache price** instead of a cost figure. Select an agent to open its **Cache** tab.
- **Needs attention**: the **Usage and cost** card surfaces **Caching** triage alerts for agents with cache activity in the window that experience low hit rates or steep drops compared to the previous period. Caching alerts require cache activity within the selected time window; an agent active earlier in a 7-day window doesn’t alert over a 24-hour window if it had no cache reads or writes during that 24 hours.

### Inspect agent cache metrics and version regressions

Open an agent in **Agents** and select the **Cache** tab to analyze caching performance:

- **Summary KPIs**: review the agent’s overall **Cache hit rate**, **Cache read tokens**, **Requests / day**, and **Uncached cost**.
- **Where input tokens went**: inspect the stacked bar breaking input tokens into **Read from cache**, **Written to cache**, and **Not cached**. Bucket percentages use largest-remainder rounding to total 100%, and token counts below 1,000 appear as whole numbers. Uncached tokens represent new request content that can’t be cached.
- **Hit rate by version**: compare prompt cache hit rates across SDK-declared agent versions (`agent_version`). Versions are ordered chronologically by when the catalog first observed each version. A version or period with input tokens but no cache reads displays as a 0% line and table row rather than missing data. A hit rate drop of 10 percentage points or more against the preceding version is flagged as **Possible regression**.
- **Miss reasons and findings**: review provider-reported cache miss reasons and automated findings, such as version churn caused by volatile system prompts. Select **Fix with your coding agent** on supported findings to copy remediation guidance for your AI assistant without exposing conversation content.

### Investigate conversation history changes

When a conversation history unexpectedly changes between turns, the LLM provider can’t reuse cached prompt prefixes. Agent Observability identifies where history diverged and whether the change affected prompt caching.

To avoid false alarms, Agent Observability only flags requests where the change costs prompt caching:

- **Caching agents only**: agents with no cache reads or writes anywhere in the conversation aren’t flagged.
- **Cache read threshold**: requests that read at least 90% of the previous request’s prompt from cache aren’t flagged. Flagged requests that achieved a partial read display the percentage still read.
- **Normalized comparisons**: tool arguments and results encoded as JSON are compared by value with sorted keys. Whitespace and key order differences aren’t treated as changes. Provider-dropped content, including thinking blocks from earlier turns and internal metadata, is excluded from comparison.

When history changes break caching, Agent Observability identifies the pattern:

Expand table

| Pattern                                      | Description                                                                                                                                                                 |
|----------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Previous request’s last message removed      | The last message from the previous turn was removed, for example, a temporary status note.                                                                                  |
| Oldest messages dropped                      | Earlier conversation turns were pruned to manage context window limits.                                                                                                     |
| Opening edited                               | Early messages or system instructions were modified.                                                                                                                        |
| Trimmed, summarized, or inserted             | Context was condensed or intervening messages were injected.                                                                                                                |
| Whitespace changed                           | Non-printing whitespace differences occurred. Visible symbols highlight characters that render invisibly (`␍` carriage return, `⇥` tab, `⍽` non-breaking space, `·` space). |
| Tool call, result, or reply sent differently | Tool arguments, tool outputs, or assistant replies varied from the previous turn.                                                                                           |
| Edited                                       | Messages were modified in place.                                                                                                                                            |

To inspect and resolve history changes in a conversation:

1. Open **Conversations** and select a conversation.
2. In the **Flow** tab header, check the request count, for example, **History changed in 2 of 15**. Click the previous and next buttons to step through affected requests.
3. In the generation view, click the **History changed · message** *N* badge to jump directly to the divergence point.
4. Review the inline **History changed here** marker. Click **Show what changed** to inspect missing, struck-through, or replaced messages.
5. Click **Fix with your coding agent** to copy a prompt describing the pattern, probable cause, and recommended code fix to paste into your coding assistant. The prompt never includes conversation text.

> Note
> 
> If you query the token metric yourself rather than reading the dashboard, check the `gen_ai_token_semantics` label first. Where it reads `inclusive`, the `input` series already contains the cache tokens, so adding `cache_read` to it counts them twice, and only `input - cache_read - cache_write - cache_creation` pays the full prompt rate.
> 
> Where the label is absent, check the provider before you add the buckets up. An absent label means nothing was declared, not that the buckets are separate: OpenAI and Gemini report an inclusive `input` whether or not the label is there. The dashboard handles both; a hand-written query has to choose.

## Reduce tool call overhead

The tools analytics page shows tool call frequency and duration. Look for:

- **Unused tools**: tools that are defined but never called add prompt tokens without value. Consider removing them.
- **Slow tools**: tools with high execution time may benefit from timeouts or caching.
- **Excessive tool calls**: agents that call too many tools per turn may need prompt adjustments.

## Attribute cost to a team, project, or developer

For SDK instrumentation, set client tags to split cost by team, project, or environment. Use the `AGENTO11Y_TAGS` environment variable or the `tags` field in the client configuration:

Bash ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy

```bash
export AGENTO11Y_TAGS=team=ai,env=production
```

Client tags reach both the generation export and the OpenTelemetry metrics, so each key becomes a Prometheus label named `agento11y_tag_<KEY>`. Turn on **Break down** in the tokens and estimated cost charts and select that label to compare spend across tag values.

A cost breakdown by a custom label uses public list prices, not your configured model prices or long-context rates. Check the **Cost shown at list prices** notice before comparing these estimates with recorded totals. For details, refer to [Optimize costs](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/observe-and-troubleshoot/dashboards/#optimize-costs).

Tags that you pass on a single generation stay on that generation and never become metric labels, so they can’t drive this breakdown. For the behavior of each mechanism, refer to [Client tags](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/instrumentation/sdk/#client-tags).

Coding-agent integrations can resolve user, repository, and branch labels automatically. For setup instructions, refer to [Attribute cost to a user, repository, or branch](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/get-started/instrument-coding-agents/#attribute-cost-to-a-user-repository-or-branch).

## Compare agent versions

When you change prompts or tools, use the agent catalog to compare the new version’s cost and performance against the previous version. Check whether the change improved quality without significantly increasing cost, or vice versa.

## Next steps

- [Run an experiment with the SDK](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/evaluate-and-improve/run-an-experiment)
- [Set up and monitor online evaluation](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/evaluate-and-improve/evaluation)
- [Configure your model prices](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/observe-and-troubleshoot/model-prices)