Optimized machine learning inference lead to reduced costs and better user experience.
Profilers are essential tools for optimizing and troubleshooting application speed, latency and resource consumption. They help reduce computation costs, fix performance issues and improve user experience. Such improvements benefit machine learning profoundly; inference latency could be reduced resulting in significantly lower costs and improved user experience.
Similar to traditional profilers, a machine learning profiler provides execution statistics, however, the focus is on ML operations and compute kernels instead of plain method calls. Additionally, ML profilers provide GPU utilization information relevant in machine learning context.
TensorFlow and PyTorch provide built-in ML profilers, which utilize NVIDIA® CUDA® profiling interface (CUPTI) under the hood for GPU profiling. One way to use those profilers is via locally installed TensorBoard. In turn, Graphsignal uses the built-in profilers as well as other tools to enable automatic profiling in any environment without installing additional software. It also allows teams to share and collaborate online.