---
title: "Monitor agents with analytics | Grafana Cloud documentation"
description: "Monitor agent activity, performance, cost, and quality using the Agent Observability analytics dashboards."
---

> For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt).

# Monitor agents with analytics

Agent Observability includes built-in analytics that visualize agent activity, performance, cost, and quality. The analytics views use Prometheus metrics and Agent Observability data to surface actionable insights.

## Before you begin

- Configure your agent to export generations and OpenTelemetry metrics to Grafana Cloud. For instructions, refer to [Get started with Grafana Cloud](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/get-started/grafana-cloud).
- Select a time range that includes your agent traffic.

## Access dashboards

Select **Analytics** in Agent Observability. Under **Explore by area**, expand:

- **Agents**: generation counts, latency, time to first token, errors, tokens, and estimated cost by agent.
- **Usage and cost**: token usage, estimated cost, cache efficiency, and conversation activity.
- **Evaluations**: evaluation scores, pass rates, and quality trends.
- **Tools**: tool call frequency, duration, and error rates.

Use the time picker to set the query window. The refresh picker lets you set an auto-refresh interval, for example, 30s, 1m, or 5m. The selected interval is saved for the session.

## Find items that need attention

The **Needs attention** section highlights agents, conversations, tools, and evaluations that may need review. Open **Health overview settings** and select **Sudden changes** to compare with the previous time window, or **High levels** to check absolute values. Turn on **Show only entities that need attention** to hide healthy cards.

Open a card to investigate its supporting data in the relevant Agent Observability view.

## Include or exclude traffic

Use the **Provider**, **Model**, and **Agent** filters to focus on specific traffic. Select one or more values in each menu, or enter a custom value. By default, the filter includes the selected values.

To hide those values instead, select **Exclude selected** in that filter’s menu. Each filter has its own mode, so you can include one provider while excluding a model or agent. Clear **Exclude selected** to return to inclusion mode.

Close the menu or click outside the dropdown to apply your selection. Selected filter chips stay visible while the URL updates, and your selections stay in the dropdown options list across data reloads. To remove values, remove chips directly or clear the filter.

Dropdown options cascade to reflect your selections:

- Selecting a provider limits **Model** options to models observed for that provider.
- Selecting a provider and model limits **Agent** options to agents that match both selections.

In coding-agent analytics views, **Provider**, **Model**, and **Agent** dropdown options are scoped to metrics from known coding-agent launchers. **Repository**, **User**, and added label filters use the same coding-agent scope. This prevents unrelated application names or unused provider and model combinations from appearing. Other analytics pages provide unscoped options across all agents.

These controls also apply when you browse conversations or review evaluation results. Your selections and exclusion modes stay in the URL when you share the view.

## Set time ranges in page URLs

When you share an Agent Observability view or link to it from a Grafana dashboard, you can control the active time range with the `from` and `to` URL query parameters.

URL time parameters accept the following formats:

- **Epoch milliseconds**: Numbers representing milliseconds since the Unix epoch, for example, `?from=1790812800000&to=1792972800000`. Grafana dashboard links and panel data links write this format when you use the `${__from}` and `${__to}` variables.
- **Relative expressions**: Relative time strings, for example, `?from=now-7d&to=now` or `?from=now-1h&to=now`.
- **ISO timestamps**: ISO dates and timestamps, for example, `?from=2026-10-01T00:00:00Z&to=2026-10-02T00:00:00Z` or `?from=20261001&to=20261002`.

If a URL parameter contains an invalid or unreadable time value, the page falls back to its default range instead of failing to load. Unreadable values aren’t saved to your session.

## Break a chart down by one or more dimensions

Each chart has a **Break down** switch that splits a single line into one line per dimension. Turn it on and choose dimensions from the picker next to it.

The picker lists common dimensions such as **Agent**, **Model**, and **Provider** first, followed by every other Prometheus label present on your `gen_ai_client_*` metrics in the current time range. Both kinds work the same way, so you can compare staging against production, or one team’s spend against another’s.

Select more than one dimension to see their combinations. Grouping by **Agent** and `deployment_environment` gives you one line per agent per environment, named `checkout-agent / production`.

You can combine up to three dimensions. Each one multiplies the number of lines on the chart, so the picker stops offering more once you reach the limit. If a chart becomes hard to read, remove a dimension or narrow the filters above it.

A label appears in the picker only if your SDK or collector attaches it to the metrics. The simplest way to add one is to set resource attributes on the service that emits the telemetry:

Bash ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy

```bash
export OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production,team=ai
```

Dots become underscores in Prometheus label names, so `deployment.environment` appears as `deployment_environment`.

You can also set client tags in the SDK, either with the `AGENTO11Y_TAGS` environment variable or the `tags` field in the client configuration:

Bash ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy

```bash
export AGENTO11Y_TAGS=team=ai,env=production
```

Each client tag becomes a label named `agento11y_tag_<KEY>`, for example, `agento11y_tag_team`. The SDK also stores the value on every generation.

For coding-agent sessions, refer to [Attribute cost to a user, repository, or branch](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/get-started/instrument-coding-agents/#attribute-cost-to-a-user-repository-or-branch) to add user, repository, and branch labels automatically.

Keep the number of distinct values small, because each value adds a metric series. For the behavior of each mechanism, refer to [Client tags](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/instrumentation/sdk/#client-tags).

Tags that you pass on a single generation, and generation metadata, aren’t metric labels, so they don’t appear in the picker. Filter conversations by `metadata.<KEY>` instead.

Your selection is remembered per chart area.

## Identify performance issues

Use the performance dashboard to spot problems:

- **High latency**: filter by agent or model to find slow generations. Drill into traces for specific conversations to identify bottlenecks.
- **Error spikes**: the error rate panel shows failures over time. Click through to conversations with errors to inspect the `call_error` payload.
- **Slow time to first token**: for streaming agents, the TTFT panel reveals which models or prompts have poor streaming performance.

## Inspect tool failures

On the **Tools** page (**Analytics** &gt; **Tools**), review tool executions, error rates, and latency.

Select **Error count** to chart the number of failed tool executions instead of the percentage shown by **Error rate**. You can also select **Error count** in the **Tools** area under **Analytics**; its table ranks tools by the highest error count.

Click **Tool**, **Executions**, **Exec %**, **Errors**, **Error rate**, or **Latency** to sort the table. Click the same header again to reverse the order. The sort menu and column headers control the same ordering.

## Optimize costs

The tokens and cost dashboard helps you find optimization opportunities:

- **Cost by model**: compare estimated cost across models and providers. Consider switching expensive calls to cheaper models where quality is acceptable.
- **Cache efficiency**: the cache read ratio shows how effectively prompt caching reduces token usage. Low cache rates may indicate prompts that change too frequently.
- **Token usage trends**: spot unexpected increases in token usage that may indicate prompt regression or unnecessary verbosity.

Select **Cache hit rate** in **Usage and cost** to compare prompt cache reuse over time. Break down by **Agent**, **Model**, or **Provider** to find differences. If you select **Token type**, the chart shows cache read and cache write token counts instead of a percentage. The conversations table also shows **Cache hit rate** so you can inspect individual sessions.

The overview cost total covers your selected time range. It uses recorded per-request costs where available and estimates earlier periods from token usage and catalog prices. **Agents** totals and **Needs attention** cost charts use the same approach. Recorded costs include your configured model prices. If no recorded costs are available, Agent Observability estimates the whole range. Token-based estimates don’t account for per-request long-context pricing tiers and can understate costs for long prompts.

Cost charts mark the transition from token-based estimates to recorded costs. A step at that marker doesn’t necessarily mean spending changed. Changing chart time buckets doesn’t change the cost total for the selected range.

When **Cost shown at list prices** appears, your selected filter or breakdown requires token-based estimates instead of recorded costs. Configured model prices and long-context pricing don’t apply to these estimates. A notice below the filters applies to the cost totals and charts on the page, not to the cost recorded for each conversation; a notice on a chart applies only to that breakdown. Remove the named filter or breakdown to return to recorded costs where available. For details, refer to [Configure your model prices](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/observe-and-troubleshoot/model-prices).

When you ask Grafana Assistant about costs from **Analytics** or **Agents**, it receives the page’s cost source and selected scope. Recorded costs include your configured prices. Estimates for older periods or unsupported labels use public list prices instead; check which source the answer uses before comparing figures.

## Track quality

The quality dashboard visualizes evaluation scores alongside operational metrics:

- **Score trends**: monitor if quality improves or degrades after agent version changes.
- **Score distributions**: identify if responses cluster around high or low scores.
- **Correlation**: compare quality scores with latency and cost to find the right balance.

## Use Prometheus metrics directly

If you need custom dashboards, query the Agent Observability OpenTelemetry metrics in Prometheus:

Expand table

| Metric                                                | Description                                                |
|-------------------------------------------------------|------------------------------------------------------------|
| `gen_ai_client_operation_duration_seconds`            | LLM call duration histogram.                               |
| `gen_ai_client_operation_time_to_first_chunk_seconds` | Streaming latency histogram for the Go `otel` export mode. |
| `gen_ai_client_time_to_first_token_seconds`           | Streaming latency histogram for all other SDK modes.       |
| `gen_ai_client_token_usage`                           | Token consumption histogram.                               |
| `gen_ai_client_tool_calls_per_operation_count`        | Tool calls per generation.                                 |

For custom cost queries, use these stack-scoped metrics from your configured Prometheus data source:

Expand table

| Metric                                    | Description                                                                               |
|-------------------------------------------|-------------------------------------------------------------------------------------------|
| `agento11y_generation_cost_usd_total`     | Cumulative estimated generation cost in USD, calculated when generations arrive.          |
| `agento11y_model_price_usd_per_1k_tokens` | Observed price in USD per 1,000 tokens, by provider, model, token type, and context tier. |
| `agento11y_model_price_request_usd`       | Observed fee in USD per request, by provider and model.                                   |

Multiplying aggregated token usage by current observed prices doesn’t reproduce recorded costs: historical rates, long-context tiers, and per-request fees can differ. Cost estimates aren’t provider invoices.

## Reuse dashboard panels

On panels with **More actions**, click **Open in Explore** to open the Prometheus query with the selected time range. Click **Add panel to dashboard** to copy the panel into a Grafana dashboard. Adding panels requires a configured Prometheus data source, dashboard create or write permission, and Grafana’s add-to-dashboard integration.

Some panels also offer **Create alert** or links to related conversations and agents. Alert creation requires Grafana Alerting, alert-creation permission, and Grafana’s create-alert integration.

## Ask scoped analytics questions

When Grafana Assistant is available, ask about the analytics page’s selected time range and filters, for example, cost, latency, errors, or tool usage. The page supplies this scope and its query context to Assistant.

A conversation can contain several model calls. Specify whether you want conversation totals or model-call totals so the answer uses the measure you need.

## Open conversations from exemplars

Dashboard exemplars connect metric spikes to example conversations. When exemplars are enabled, histogram panels can show exemplar markers with links that open the matching conversation.

Use exemplar links to move from a latency or error spike to the conversation that produced it. This feature requires exemplars to be enabled for your stack.

## Set up alerts

Create Grafana alerts on Agent Observability metrics to proactively catch issues:

- Alert on error rate exceeding a threshold.
- Alert on p95 latency exceeding SLO targets.
- Alert on estimated cost per day exceeding budget.
- Alert on evaluation score drops below a quality threshold.

Configure alerts in Grafana using the standard alerting workflow with the Prometheus data source.

## Schedule recurring reports

Use reports to receive recurring summaries of your agents’ cost, latency, errors, or quality. You need Grafana Assistant available in your stack and permission to create automations.

1. On the Agent Observability analytics page, click **Add report**.
2. Describe the summary you want, for example, a weekly cost report, then click **Continue in Assistant**. You can also select a template under **Starting points**.
3. Review the report instructions, schedule, and delivery settings in Grafana Assistant, then create the automation.

Describing a report or selecting a template opens a draft; it doesn’t schedule a report until you create the automation. Click **Manage automations** to manage schedules, delivery, and run history in Grafana Assistant.

The **Weekly cost digest** compares recorded costs for the last seven days with the previous seven days. These costs use your configured model prices where available and catalog prices otherwise. Unlike dashboard totals, this report doesn’t estimate older periods from token usage. If it can’t read recorded costs, it reports **No data** instead of estimating spend.

## Next steps

- [Optimize cost and performance](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/observe-and-troubleshoot/cost-optimization)
- [Browse and debug conversations](https://grafana.com/docs/grafana-cloud/observe-and-act/agent-observability/observe-and-troubleshoot/conversations)