Batching and Performance

What you’ll learn¶

Understand the two-level batching model (artifacts → execution units → workers)
Configure ExecutionConfig fields for different workload patterns
Measure the impact of batching on pipeline performance
Choose batching parameters based on workload characteristics

Prerequisites: First Pipeline.
Estimated time: 20 minutes
GPU required: No.

By default, each artifact gets its own execution unit. This tutorial shows how to group work using artifacts_per_unit, explains the full two-level batching model, and develops intuition for tuning ExecutionConfig fields.

from __future__ import annotations

from artisan.operations.examples import (
    DataGenerator,
    DataTransformer,
    MetricCalculator,
)
from artisan.orchestration import Backend, PipelineManager
from artisan.utils import tutorial_setup
from artisan.visualization import build_micro_graph, inspect_pipeline

env = tutorial_setup("batching_performance")

The two-level batching model¶

Artisan batches work in two stages:

Level 1: Artifacts → Execution units. Controlled by artifacts_per_unit. Each execution unit is one call to the operation’s execute() method.

Level 2: Execution units → Workers. Controlled by units_per_worker and max_workers. Each worker is one process (local) or one SLURM job (submitted). Multiple units can run sequentially within one worker.

ExecutionConfig fields¶

The two most important fields are artifacts_per_unit (how many artifacts go into each execution unit) and units_per_worker (how many units each worker processes sequentially). For the full list of ExecutionConfig fields, see Configuring Execution.

Pass these as a dict to the execution parameter of pipeline.run().

Baseline: default batching¶

First, run a pipeline with the default configuration (1 artifact per unit) to establish a timing baseline.

pipeline = PipelineManager.create(
    name="baseline",
    delta_root=env.delta_root,
    staging_root=env.staging_root,
    working_root=env.working_root,
)
output = pipeline.output

pipeline.run(
    operation=DataGenerator,
    name="generate",
    params={"count": 4, "seed": 42},
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=DataTransformer,
    name="transform",
    inputs={"dataset": output("generate", "datasets")},
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=MetricCalculator,
    name="metrics",
    inputs={"dataset": output("transform", "dataset")},
    backend=Backend.LOCAL,
)

pipeline.finalize()
inspect_pipeline(env.delta_root)

build_micro_graph(env.delta_root)

With default batching, the transform step creates 4 execution units — one per artifact. Each grey box in the micro graph is a separate execution record. Every artifact gets its own invocation of the operation.

Tuning `artifacts_per_unit`¶

Group multiple artifacts into fewer execution units to reduce per-unit overhead. This is the most common tuning parameter.

env_apu = tutorial_setup("batching_apu")

pipeline = PipelineManager.create(
    name="apu_tuning",
    delta_root=env_apu.delta_root,
    staging_root=env_apu.staging_root,
    working_root=env_apu.working_root,
)
output = pipeline.output

pipeline.run(
    operation=DataGenerator,
    name="generate",
    params={"count": 4, "seed": 42},
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=DataTransformer,
    name="transform",
    inputs={"dataset": output("generate", "datasets")},
    execution={"artifacts_per_unit": 2},
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=MetricCalculator,
    name="metrics",
    inputs={"dataset": output("transform", "dataset")},
    execution={"artifacts_per_unit": 4},
    backend=Backend.LOCAL,
)

pipeline.finalize()
inspect_pipeline(env_apu.delta_root)

build_micro_graph(env_apu.delta_root)

Compare this micro graph to the baseline. The transform step now has 2 execution records (grey boxes) instead of 4 — each one processes 2 artifacts. The metrics step has 1 execution record (4 artifacts / 4 per unit). Fewer execution units means less overhead from process creation and artifact serialization.

Tuning `units_per_worker`¶

On SLURM clusters, each worker is a separate job submission. Job startup has significant overhead (queue wait, container launch, environment setup). units_per_worker packs multiple execution units into a single job to amortize this cost.

Locally, units_per_worker packs units into a single process. This reduces process creation overhead but runs the units sequentially within that process.

env_upw = tutorial_setup("batching_upw")

pipeline = PipelineManager.create(
    name="upw_tuning",
    delta_root=env_upw.delta_root,
    staging_root=env_upw.staging_root,
    working_root=env_upw.working_root,
)
output = pipeline.output

pipeline.run(
    operation=DataGenerator,
    name="generate",
    params={"count": 4, "seed": 42},
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=DataTransformer,
    name="transform",
    inputs={"dataset": output("generate", "datasets")},
    execution={"artifacts_per_unit": 2, "units_per_worker": 2},
    backend=Backend.LOCAL,
)

pipeline.finalize()
inspect_pipeline(env_upw.delta_root)

build_micro_graph(env_upw.delta_root)

With artifacts_per_unit=2 and 4 inputs, we get 2 execution units. With units_per_worker=2, both units are packed into 1 worker. On SLURM, this means 1 job instead of 2. Locally, it means 1 process running both units sequentially.

The primary benefit is on SLURM: fewer job submissions means less queue wait time and container startup overhead. Locally the benefit is smaller — it mainly reduces process creation cost.

Operation-level defaults¶

Operations can declare their own ExecutionConfig defaults in the class definition. Step-level overrides take precedence.

class MyExpensiveOperation(OperationDefinition):
    name = "my_expensive_op"

    # Declare default batching for this operation
    execution = ExecutionConfig(
        artifacts_per_unit=10,
        max_workers=4,
    )

When you call pipeline.run(MyExpensiveOperation) without an execution override, these defaults are used. If you pass execution={"max_workers": 2}, the step override wins.

Tuning guidelines¶

Workload type	`artifacts_per_unit`	`units_per_worker`	`max_workers`
Fast, many artifacts (e.g., metric computation)	10-50	1	Default
Slow, few artifacts (e.g., expensive per-item processing)	1	1	Limited by GPU count
SLURM with container startup	5-20	3-10	Default
Memory-intensive	1-5	1	Limited by available memory
I/O-bound	10-100	1	Default (I/O parallelism helps)

Rules of thumb:

Start with defaults and use build_micro_graph to see how work is grouped
If each execution handles too few artifacts, increase artifacts_per_unit
If SLURM queue wait dominates, increase units_per_worker
If resource contention is an issue, decrease max_workers
Use PipelineTimings from the Timing Analysis tutorial to measure exact phase durations

Summary¶

Concept	Parameter	Effect
Artifact grouping	`artifacts_per_unit`	Fewer execution units, less per-unit overhead
Unit cap	`max_artifacts_per_unit`	Prevents oversized units with variable inputs
Job packing	`units_per_worker`	Fewer SLURM jobs, amortized startup cost
Parallelism limit	`max_workers`	Caps concurrent processes/jobs for resource control
Time hint	`estimated_seconds`	SLURM time allocation (no effect locally)
Job naming	`job_name`	Custom SLURM job name for monitoring

Key takeaway: Start with defaults, use build_micro_graph to see how work is grouped, and tune the parameter that targets your specific bottleneck. artifacts_per_unit is almost always the first knob to turn.

Next steps¶

Step Overrides — All pipeline.run() override parameters
Timing Analysis — Diagnose performance with timing DataFrames
Configuring Execution — Complete execution configuration reference