LlamaIndex Tracing And Monitoring
See the Quick Start guide on how to install and configure Graphsignal.
Graphsignal supports instrumenting, tracing and monitoring LlamaIndex. Traces and metrics provide execution details for each retrieval. These insights include prompts, completions, embedding statistics, retrieved nodes, parameters, latency, and exceptions.
When OpenAI APIs are used, Graphsignal automatically instruments and traces OpenAI API providing additional insights such as token counts and costs.
Integration
Add Graphsignal callback handler:
from graphsignal.callbacks.llama_index import GraphsignalCallbackHandler
callback_manager = CallbackManager([GraphsignalCallbackHandler()])
Settings.embed_model = TestEmbedding()
Settings.callback_manager = callback_manager
index = GPTVectorStoreIndex.from_documents(documents, embed_model=Settings.embed_model)See LlamaIndex Callbacks documentation for more information.