Tracing and Monitoring Local Models

Integration

Graphsignal tracer can be used in model applications that run models locally. See Quick Start guide on how to install and configure Graphsignal tracer.

Model deployments, versions or environments can be tracked separately. Simply set a different name to deployment in configure method.

import graphsignal

graphsignal.configure(api_key='my-api-key', deployment='my-app-prod')

def predict(x):
    with graphsignal.start_trace('predict') as span:
        y = model(x)
        span.set_data('input', x)
        span.set_data('output', y)

or using decorator:

@trace_function
def predict(x):
    # model inference here

To record additional information with traced inferences, provide tags to graphsignal.start_trace() method or use Trace.set_tag method.

To track data metrics and record per-inference data samples, Trace.set_data() method can be used.

with graphsignal.start_trace('predict') as span:
    span.set_data('input', input_data)

For model servers that only accept model files and expose them via REST API, see Local Models guide.