macrostat.util.batchprocessing

Batch processing functionionality

Functions

parallel_processor([tasks, worker, cpu_count])	Run all of the tasks in parallel using the ProcessPoolExecutor.
pool_context(*args, **kwargs)	Context manager for process pool to ensure proper cleanup.
timeseries_worker(task)	Worker function for parallel_processor, which will execute a simulation with the given parameters and return the output.

macrostat.util.batchprocessing.parallel_processor(tasks: list = [], worker: callable = <function timeseries_worker>, cpu_count: int = 1)

Run all of the tasks in parallel using the ProcessPoolExecutor.

macrostat.util.batchprocessing.pool_context(*args, **kwargs)

Context manager for process pool to ensure proper cleanup.

macrostat.util.batchprocessing.timeseries_worker(task: tuple)

Worker function for parallel_processor, which will execute a simulation with the given parameters and return the output.

Parameters:

task (tuple) – Tuple of (name, model, *args) where name is the name of the simulation, model is the model to be simulated and *args are the arguments to be passed to the model’s simulate method.

Returns:

Tuple of (name, *args, output) where name is the name of the simulation, *args are the arguments passed to the model’s simulate method and output is the output of the simulation.

Return type:

tuple