GPU profiler for AI agents to autonomously optimize inference performance across models, engines, and GPUs.
The engine decodes faster than the baseline (6.8 vs 8.8 ms/token) but loses on end-to-end agent trajectories. Prefill is most of the gap. Profiling a replay of the longest trajectory, a ~30K-token cold prefill, under graphsignal-run.
⎻ span 29.6 s, 0 errors, GPU utilization 99.8%, no clock throttling
⎻ kernels, cumulative over 19.4 s:
attn_prefill_kernel 26.8%
gemm_bf16mma<128,128,64> 37.0% (2 instantiations)
gdn_chunk_scan + wy_prep 12.2%
attn_pv + attn_scores 8.4% decode verify path
gemv_kernel<…> 9.5% decode
decode graphs: 21.0 / 19.3 ms per step Prefill is 76% of kernel time; decode is already fine. The GEMM runs at ~194 TFLOP/s at M = 512 and split-K stopped helping there, so the lever is more rows per launch. Trying a 1024-token prefill chunk.
⎻ Σ T_traj 57.27 s vs 60.60 s (−5.5%); prefill −7.9% ⎻ token stream and KV digests identical to chunk 512
Same bits, 5.5% off the trajectory. Rebuilding with it as the default and re-running the full 13-trajectory benchmark. Next target: the multi-query attention on the verify path, 8.4% here and 11.4 ms per round at 32K…
AMD
PyTorch
vLLM
SGLang
TensorRT
Inference profiling
Cumulative time per kernel, per CUDA graph, and per memory or synchronization operation via CUPTI and ROCm, plus GPU telemetry, engine metrics, and error capture.
GPU probes
A vendorable C++ header that you — or an AI agent — use to instrument application code and CUDA/HIP kernels. Lock-free recording, inert without the profiler — probe values appear next to the built-in profiles and metrics.
Built for AI agents
One local JSON endpoint serves everything measured. An agent launches the workload, reads what dominates, changes flags or code, and measures again — entirely on your machine.
Production feedback loop
Optionally, signals upload to Graphsignal, and agents read production behavior back via the Signals API to tune the next deployment. Without an API key, no profiling data is ever uploaded.