LlamaIndex Tracing And Monitoring

See the Quick Start guide on how to install and configure Graphsignal.

Graphsignal supports instrumenting, tracing and monitoring LlamaIndex. Traces and metrics provide execution details for each retrieval. These insights include prompts, completions, embedding statistics, retrieved nodes, parameters, latency, and exceptions.

When OpenAI APIs are used, Graphsignal automatically instruments and traces OpenAI API providing additional insights such as token counts and costs.


Add Graphsignal callback handler:

from graphsignal.callbacks.llama_index import GraphsignalCallbackHandler

callback_manager = CallbackManager([GraphsignalCallbackHandler()])

Settings.embed_model = TestEmbedding()
Settings.callback_manager = callback_manager

index = GPTVectorStoreIndex.from_documents(documents, embed_model=Settings.embed_model)

See LlamaIndex Callbacks documentation for more information.