See the Quick Start guide on how to install and configure Graphsignal.
Graphsignal automatically instruments and monitors Banana Python SDK. Besides reporting latency, throughput and utilization, Graphsignal also records data statistics for input and output data, and automatically records latency outliers and exceptions.
Graphsignal can also monitor and optionally profile custom model inference code running in Docker containers on Banana servers. Simply use
import graphsignal # The API key is provided via environment variables GRAPHSIGNAL_API_KEY graphsignal.configure(deployment='my-model-prod') ... @graphsignal.trace_function def inference(model_inputs:dict) -> dict: # model inference code here
Or wrap the inference code with
start_trace in order to record more information in the trace, such as parameters and data statistics:
with graphsignal.start_trace(endpoint='generate') as trace: output = model.generate(prompt) trace.set_data('prompt', prompt) trace.set_data('output', output)
The API key can be set in in Banana UI in deployment's settings as build argument and then assigned to environment variable in the
ARG GRAPHSIGNAL_API_KEY ENV GRAPHSIGNAL_API_KEY=$GRAPHSIGNAL_API_KEY
GPU and process metrics are automatically recorded. If any supported framework is used, e.g. PyTorch, traces will include more insights, such as memory allocations. Similarly to Banana client monitoring, Graphsignal automatically records latency outliers and exceptions, and optionally data issues, such as missing values.
The Banana Client Example illustrates how to add Graphsignal agent to applications that use Banana Python SDK.
The Banana Server Example illustrates how to add Graphsignal agent to Banana templates.