Skip to main content

scorebook.metrics.metric_registry

Registry module for evaluation metrics.

This module maintains a centralized registry of available evaluation metrics that can be used to assess model performance. It provides a single access point to retrieve all implemented metric classes.

MetricRegistry Objects​

class MetricRegistry()

A registry for evaluation metrics.

This class provides a central registry for all evaluation metrics in the system. It allows metrics to be registered with unique names and retrieved either by name or by class. The registry ensures that metrics are properly initialized and accessible throughout the application.

The registry supports:

  • Registering new metric classes with optional custom names
  • Retrieving metric instances by name or class
  • Listing all available metrics

Usage: @MetricRegistry.register("custom_name") class MyMetric(MetricBase): ...

Get by name

metric = MetricRegistry.get("custom_name")

Get by class

metric = MetricRegistry.get(MyMetric)

List available metrics

metrics = MetricRegistry.list_metrics()

register​

@classmethod
def register(cls) -> Callable[[Type[MetricBase]], Type[MetricBase]]

Register a metric class in the registry.

Returns:

A decorator that registers the class and returns it.

Raises:

  • ValueError - If a metric with the given name is already registered.

get​

@classmethod
def get(cls, name_or_class: Union[str, Type[MetricBase]],
        **kwargs: Any) -> MetricBase

Get an instance of a registered metric by name or class.

Arguments:

  • name_or_class - The metric name (string) or class (subclass of BaseMetric).
  • **kwargs - Additional arguments to pass to the metric's constructor.

Returns:

An instance of the requested metric.

Raises:

  • ValueError - If the metric name is not registered.

list_metrics​

@classmethod
def list_metrics(cls) -> List[str]

List all registered metrics.

Returns:

A list of metric names.