graphsignal-top: a live view of your inference GPUs

The open-source Graphsignal profiler serves everything it measures as JSON for agents. graphsignal-top is the same data as a live terminal screen: GPU telemetry, engine throughput and latency, and kernels ranked by time, locally or over the network.

Graphsignal vs. torch.profiler and nsys

torch.profiler and nsys already give you per-kernel GPU time, so what does Graphsignal add? Comparable JSON snapshots, telemetry in the same document, GPU probes, overhead low enough to run in production, and a runner that wraps any engine.

Understanding Inference for Systems Engineers

A stage-by-stage walk through what actually happens when a large language model generates text, told through one real engine built for one model on one GPU: tokenization, the forward pass layer by layer, fp8 weights, prefill, CUDA graphs and megakernels, sessions, speculative decoding, grammar-constrained decoding, and correctness.

CUDA Profiler for Production Inference

Why dev-time CUDA profilers don't fit production inference, and what a profiler built for it looks like: low-overhead kernel attribution, host sync waits, and integrated telemetry.

vLLM Production Observability: From Model to Hardware

Production-grade profiling and monitoring for vLLM: always-on vLLM, PyTorch and CUDA profiling with tracing, metrics and errors in one place.