Transformers Agents In Production

Hugging Face Transformers Agents are a natural evolution and an abstraction for models hosted on Hugging Face. It is a simple and elegant implementation of an AI agent.

When applications relying on Transformers Agents are deployed to production and serve user requests, a full visibility into agent runs is necessary to ensure applications behave as expected and users have the best experience. The essential signals include:

Agent decisions and explanations.
Tool inputs and outputs.
Run latency breakdown by operations.
Errors and exceptions.
Outliers.
Compute utilization.

Tracing Agent Runs With Graphsignal

Graphsignal automatically instruments and starts tracing and monitoring Transformers Agents. It's only necessary to set it up by providing Graphsignal API key and a deployment name.

import graphsignal

# Provide an API key directly or via GRAPHSIGNAL_API_KEY environment variable
graphsignal.configure(api_key='my-api-key', deployment='my-langchain-app-prod')

You can get an API key here.

In order to additionally trace other functions, e.g. request handlers, you can also use a decorator or context manager. See the Quick Start for complete setup instructions.

Here is a simple example demonstrating how to add the Graphsignal tracer. See the complete example here.

After the application is started, traces and metrics are automatically recorded and available in the dashboards.

Tracking OpenAI API Costs

If OpenAI LLM is used as an agent, Graphsignal automatically instruments and traces the OpenAI API providing additional insights such as costs, token counts, sizes, and finish reasons.

Learn more about OpenAI cost tracking here.

Performance, Data, and System Metrics

Performance, data metrics, and resource utilization are available, to monitor applications over time and correlate any changes or issues.

Alerts can be set up to get notified on exceptions or outliers.

Give it a try and let us know what you think. Follow us at @GraphsignalAI for updates.