Skip to main content

scorebook.utils.progress_bars

Progress bar utilities for evaluation tracking.

EvaluationProgressBars Objects​

class EvaluationProgressBars()

Manages nested progress bars for evaluation tracking.

__init__​

def __init__(datasets: List[Any], hyperparam_count: int, parallel: bool,
             total_eval_runs: int) -> None

Initialize progress bar manager.

Arguments:

  • datasets - List of datasets being evaluated
  • hyperparam_count - Number of hyperparameter configurations per dataset
  • parallel - Whether running in parallel mode
  • total_eval_runs - Total number of EvalRunSpecs (dataset_count * hyperparam_count)

start_progress_bars​

def start_progress_bars() -> None

Start both progress bars.

on_eval_run_completed​

def on_eval_run_completed(dataset_idx: int) -> None

Update progress when an eval run (EvalRunSpec) completes in parallel mode.

on_hyperparam_completed​

def on_hyperparam_completed(dataset_idx: int) -> None

Update progress when a hyperparameter config completes in sequential mode.

close_progress_bars​

def close_progress_bars() -> None

Close both progress bars.

evaluation_progress​

@contextmanager
def evaluation_progress(
        datasets: List[Any], hyperparam_count: int, parallel: bool,
        total_eval_runs: int) -> Generator[EvaluationProgressBars, None, None]

Context manager for evaluation progress bars.

Arguments:

  • datasets - List of datasets being evaluated
  • hyperparam_count - Number of hyperparameter configurations per dataset
  • parallel - Whether running in parallel mode
  • total_eval_runs - Total number of EvalRunSpecs

Yields:

  • EvaluationProgressBars - Progress bar manager instance