Banana Monitoring

See the Quick Start guide on how to install and configure Graphsignal.

Client SDKs

Graphsignal automatically instruments and monitors Banana Python SDK. Besides reporting latency, throughput and utilization, Graphsignal also records data statistics for input and output data, and automatically records latency outliers and exceptions.

Inference server

Graphsignal can also monitor and optionally profile custom model inference code running in Docker containers on Banana servers. Simply use trace_function decorator:

import graphsignal

# The API key is provided via environment variables GRAPHSIGNAL_API_KEY
graphsignal.configure(deployment='my-model-prod')

...

@graphsignal.trace_function
def inference(model_inputs:dict) -> dict:
    # model inference code here

Or wrap the inference code with start_trace in order to record more information in the trace, such as parameters and data statistics:

with graphsignal.start_trace(endpoint='generate') as trace:
    output = model.generate(prompt)
    trace.set_data('prompt', prompt)
    trace.set_data('output', output)

The API key can be set in in Banana UI in deployment's settings as build argument and then assigned to environment variable in the Dockerfile:

ARG GRAPHSIGNAL_API_KEY
ENV GRAPHSIGNAL_API_KEY=$GRAPHSIGNAL_API_KEY

GPU and process metrics are automatically recorded. If any supported framework is used, e.g. PyTorch, traces will include more insights, such as memory allocations. Similarly to Banana client monitoring, Graphsignal automatically records latency outliers and exceptions, and optionally data issues, such as missing values.

Examples

The Banana Client Example illustrates how to add Graphsignal agent to applications that use Banana Python SDK.

The Banana Server Example illustrates how to add Graphsignal agent to Banana templates.