Model Serving Monitoring and Profiling

Integration

Graphsignal agent can be used in model servers that run custom Python inference code. See Quick Start guide on how to install and configure the Graphsignal agent.

Model deployments, versions or environments should be tracked separately. Simply set a different name to deployment in configure method.

import graphsignal

graphsignal.configure(api_key='my-api-key', deployment='my-app-v1-prod')

def predict(x):
    with graphsignal.start_trace(endpoint='my-model-predict'):
        # any function or code segment

To record additional information with traced inferences, provide tags to start_trace method or use EndpointTrace.set_tag method.

To track data metrics and record per-inference data profiles, EndpointTrace.set_data method can be used.

with graphsignal.start_trace(endpoint='predict') as trace:
    trace.set_data('input', input_data)

For model servers that only accept model files and expose them via REST API, client-side integration can be used.

Examples

This FastAPI example illustrates how to integrate Graphsignal.