API Reference

graphsignal.configure

configure(api_key, debug_mode=False, window_seconds=600, buffer_size=100)

Configures and initializes the logger.

Arguments:

  • api_key (str): The access key for communication with the Graphsignal servers.
  • debug_mode (bool, optional, default is False): Enable/disable debug output.
  • window_seconds (int, optional, default 300): The length of prediction time windows for which data statistics are reported.
  • buffer_size (int, optional, default 100): The maximum mumber of instances kept in memory before computing statistics.

graphsignal.session

session(deployment_name)

Get logging session for a deployed model identified by model deployment name.

with statement can be used for offline/batch logging to make sure all remaining buffered data is automatically uploaded at the end.

Arguments:

  • deployment_name (str): Model deployment name, e.g. 'model1_prod', 'model2_canary' or any other string value.

Returns:

  • Session - session object for logging prediction and evaluation data.

Raises:

  • ValueError - When arguments are missing or invalid.

graphsignal.shutdown

shutdown()

Send any collected data to the server and cleanup.

Normally, when python scripts exists, this method is automatically called. Use this method, if you want to explicitly shutdown the logger.

Session.log_metadata

log_metadata(key, value)

Log model metadata.

Arguments:

  • name (str): Metadata entry key.
  • value (str, int, float): Metadata entry value.

Raises:

  • ValueError - When arguments are missing or invalid.

Session.log_prediction

log_prediction(features=None, output=None, actual_timestamp=None)

Log single model prediction to monitor data schema changes and drift.

featues or output or both must be provided.

Data statistics are computed and uploaded for time windows at certain intervals and on process exit. No raw data is uploaded.

The maximum number of features (dict keys) is currently limited to 250. If data contains more columns, only first 250 will be processed.

Example:

sess.log_prediction(
  features={'f1': 1.2, 'f2': 'abc'},
  output=True)

Arguments:

  • features (dict, optional): Model input features.
  • output (str, bool, int, float, tuple, list, dict, optional): Model output.
  • actual_timestamp (int, optional, default is current timestamp): Actual unix timestamp of the prediction, when different from current timestamp.

Raises:

  • ValueError - When arguments are missing or invalid.

Session.log_prediction_batch

log_prediction_batch(features=None, feature_names=None, outputs=None, output_names=None, actual_timestamp=None):

Log model prediction batch to monitor data schema changes and drift.

featues or outputs or both must be provided.

Data statistics are computed and uploaded for time windows at certain intervals and on process exit. No raw data is uploaded.

The maximum number of features (columns) is currently limited to 250. If data contains more columns, only first 250 will be processed.

Supported features batch formats:

dict of list:

{'feature1': [1, 2], 'feature2': [1.1, 2.1]}

list of list (rows * features):

[[1, 2.0],[4, 15.1]]

Two-dimensional numpy.ndarray (rows * features):

numpy.asarray([[1, 3.1], [2, 4.1]])

pandas.DataFrame:

pandas.DataFrame(data=[[1, 1.1], [2, 2.1]], columns=['feature1', 'feature2'])

Supported outputs batch formats:

list of list (rows * outputs):

[[True], [True]]

list of list for multi-output predictions (rows * outputs):

[[0.1, 0.9], [0.2, 0.8]]

dict of list:

{'1': [0.1, 0.8], '0': [0.9, 0.2]}

Two-dimensional numpy.ndarray (rows * outputs):

numpy.asarray([[0.75], [0.59]])

pandas.DataFrame:

pandas.DataFrame(data=[[0.3], [0.5], [0.2]])

Arguments:

  • features (list, dict, numpy.ndarray, pandas.DataFrame, optional): Model input features batch.
  • feature_names (list, optional): A list of input data column names. If not provided, column names are inferred from features.
  • outputs (list, dict, numpy.ndarray, pandas.DataFrame, optional): Model output batch.
  • output_names (list, optional): A list of output data column names. If not provided, column names are inferred from outputs.
  • actual_timestamp (int, optional, default is current timestamp): Actual unix timestamp of the prediction, when different from current timestamp.

Raises:

  • ValueError - When arguments are missing or invalid.

Session.log_evaluation

log_evaluation(prediction, label, segments=None, actual_timestamp=None):

Log prediction and ground truth label to evaluate model performance. Because ground truth is usually available at a later point, evaluation logging is independent from prediction logging. Prediction logging is not required for model performance monitoring and visualization.

Both label and prediction should be provided and have the same type. The values must be equal if model prediction was correct.

If types are bool, model output is treated as binary.

If types are str, model output is treated as categorical.

If types are int or float, model output is treated as numeric.

Performance metrics are computed based on the model output type.

Example:

sess.log_evaluation(
  prediction=False,
  label=True, 
  segments=['seg1', 'seg2'])

Arguments:

  • prediction (str, bool, int, float): Model prediction.
  • label (str, bool, int, float): True label.
  • segments (list, optional): Allows to compute accuracy and other performance metrics for provided segment(s) in addition to model-level metrics. Segments are currently supported for binary and catigorical model outputs only.
  • actual_timestamp (int, optional, default is current timestamp): Actual unix timestamp of the evaluation or prediction, when different from current timestamp.

Raises:

  • ValueError - When arguments are missing or invalid.

Session.log_evaluation_batch

log_evaluation_batch(predictions, labels, segments=None, actual_timestamp=None):

Log predictions and ground truth labels to evaluate model performance. Because ground truth is usually available at a later point, evaluation logging is independent from prediction logging. Prediction logging is not required for model performance monitoring and visualization.

Log evaluation batch by passing predictions and labels using list or numpy.ndarray. See log_evaluation(...) method documentation for more information on array element data types.

Example:

sess.log_evaluation_batch(
  predictions=[0.3, 0.43, 0.8], 
  lables=[0.2, 0.5, 0.8],
  segments=[['seg1'], ['seg2'], ['seg2']])

Arguments:

  • predictions (list, numpy.ndarray): Model predictions.
  • labels (list, numpy.ndarray): True labels.
  • segments (list, optional): Allows to compute accuracy and other performance metrics for provided segment(s) in addition to model-level metrics. Segments are currently supported for binary and catigorical model outputs only.
  • actual_timestamp (int, optional, default is current timestamp): Actual unix timestamp of the evaluation or prediction, when different from current timestamp.

Raises:

  • ValueError - When arguments are missing or invalid.

Session.flush

flush()

Normally, logged data is pre-processed and uploaded chronologically in time windows according to log event timestamp or provided actual_timestamp. It is also uploaded on logging session context exit, if session was obtained using with graphsignal.session(...), and on process exit.

This method may be useful in some situations when uploading needs to be controlled.