Experiment Tracking, Monitoring and Profiling

Integration

Graphsignal can be used to profile inference in experiment tracking mode.

To identify each run, pass tags to start_trace method.

for sample in dataset:
    with graphsignal.start_trace(endpoint='predict', tags=dict(run_name='run1')):
        # inference code

Depending on the framework several profilers are available. By providing profiler to start_trace method, the profiler will be automatically activated for some inferences. No need to explicitly start and stop it. The following values are currently supported: python or True (default), tensorflow, pytorch, jax, onnxruntime. See integration documentation for more details.

with graphsignal.start_trace(endpoint='predict', tags=dict(run_name='run1'), profiler='pytorch'):
    # inference code

Examples

The ONNX BERT example example illustrates this use case.