Scores and Feedback

Introduction

Scores allow recording an evaluation of any event or object, such as generation, run, session, or user. Scores can be associated with events or objects using tags, but can also be set directly to a span.

Integration

See Quick Start guide on how to install and configure Graphsignal tracer.

Tag a request, run, session, or user for every request or run. Learn more on how to set session and user tags in Session Tracking and User Tracking guides.

Create a score for the tag you've set during tracing. This can be done at a later time and/or by other application, for example, when user clicks thumbs-up or thumbs-down for a request or a session.

Python

graphsignal.score('user_feedback', tags=dict('session_id', session_id), score=1, comment=user_comment)

You can also associate a score with a span directly:

with graphsignal.trace('generate') as span:
    ...
    span.score('prompt_injection', score=0.7, severity=2)

See API reference for more information on graphsignal.score and Span.score methods.

REST API

See REST API reference for more information on uploading scores.

Here is an example of using the REST API to upload scores.

curl -X POST 'https://api.graphsignal.com/api/v1/scores/' \
-H 'Content-Type: application/json' \
-H 'Authorization: YOUR_GRAPHSIGNAL_API_KEY' \
-d '[
    {
        "tags": [
            {"key": "session_id", "value": "mysessionid"}
        ],
        "name": "user_feedback",
        "comment": "user comment",
        "score": 10,
        "severity": 1,
        "create_ts": 1617958400
    }
]'

Alerting

If you'd like to get notified when a score is being created and, if required, is within certain range, you can set up alerts on Settings / Alerting page. Alerts can be delivered over email, Slack message or web hook.