Timing Analysis

What you’ll learn¶

Load pipeline timing data with PipelineTimings.from_delta()
Inspect step-level phase timings as a DataFrame
Plot stacked bar charts of step phase breakdowns
Drill into per-execution timings for individual steps
Compare mean execution timings across steps

Every pipeline step records timing information for each execution phase: resolving inputs, batching and cache checking, executing operations, committing results, and compacting tables. PipelineTimings loads this data from the delta store and provides DataFrames and plots for identifying bottlenecks.

Prerequisites: Your First Pipeline, Exploring Results.
Estimated time: 10 minutes
GPU required: No.

from __future__ import annotations

from artisan.operations.curator import Filter
from artisan.operations.examples import (
    DataGenerator,
    DataTransformer,
    MetricCalculator,
)
from artisan.orchestration import Backend, PipelineManager
from artisan.utils import tutorial_setup
from artisan.visualization.timing import PipelineTimings

env = tutorial_setup("timing_analysis")

Build a multi-step pipeline to generate timing data. We use batching on the DataTransformer step to create multiple execution units, which makes the per-execution analysis more interesting.

pipeline = PipelineManager.create(
    name="timing_tutorial",
    delta_root=env.delta_root,
    staging_root=env.staging_root,
    working_root=env.working_root,
)
output = pipeline.output

pipeline.run(
    operation=DataGenerator,
    name="generate",
    params={"count": 10, "seed": 42},
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=DataTransformer,
    name="transform",
    inputs={"dataset": output("generate", "datasets")},
    execution={"artifacts_per_unit": 5},
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=MetricCalculator,
    name="score",
    inputs={"dataset": output("transform", "dataset")},
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=Filter,
    name="filter",
    inputs={"passthrough": output("transform", "dataset")},
    params={
        "criteria": [
            {"metric": "distribution.median", "operator": "gt", "value": 0.3},
        ]
    },
    backend=Backend.LOCAL,
)
pipeline.run(
    operation=DataTransformer,
    name="refine",
    inputs={"dataset": output("filter", "passthrough")},
    params={"seed": 99},
    backend=Backend.LOCAL,
)

pipeline.finalize()

Load timing data¶

Load timing data from the delta store with PipelineTimings.from_delta(). This reads the steps and executions tables, parses the embedded timing metadata, and structures it for analysis.

timings = PipelineTimings.from_delta(env.delta_root)

Step-level timings¶

The .step_timings() method returns a DataFrame with one row per step. Each row includes the step number, name, total duration, and one column per timing phase. The phases vary by operation type — creator steps include phases like capture_logs, while curator steps do not.

timings.step_timings()

Plot step phase breakdown¶

The .plot_steps() method renders a stacked horizontal bar chart showing how each step’s time breaks down by phase. This is the fastest way to spot which steps are slow and why.

timings.plot_steps()

You can filter to specific steps with the step_numbers parameter -- useful for focusing on a subset of a long pipeline.

timings.plot_steps(step_numbers=[0, 1, 2])

Execution-level timings¶

Step-level timings tell you which step is slow. Execution-level timings tell you why — they break down each individual execution unit within a step. An execution unit is one batch of artifacts processed together — when a step uses batching, it produces multiple execution units that can run in parallel.

Note that execution-level phases differ from step-level phases. Step phases track the orchestration lifecycle (resolve inputs, batch, commit, etc.), while execution phases track the operation lifecycle (setup, execute, record, etc.).

timings.execution_timings(step_number=1)

Each row is one execution unit. Step 1 used artifacts_per_unit=5 with 10 inputs, so you should see 2 execution units.

Execution statistics¶

For steps with many execution units, summary statistics are more useful than raw timings. The .execution_stats() method computes mean, standard deviation, min, and max for each phase across all execution units in a step. Step 1 has 2 execution units, so the stats are meaningful here.

timings.execution_stats(step_number=1)

Compare execution timings across steps¶

The .plot_execution_stats() method compares mean execution timings across all steps. This helps answer: Which operation is the slowest per execution unit?

timings.plot_execution_stats()

Summary¶

Concept	API	Purpose
Load timings	`PipelineTimings.from_delta(delta_root)`	Parse timing metadata from delta tables
Step timings	`.step_timings()`	DataFrame with per-step phase breakdown
Plot steps	`.plot_steps()`	Stacked bar chart of step phases
Execution timings	`.execution_timings(step_number=N)`	Per-unit breakdown within a step
Execution stats	`.execution_stats(step_number=N)`	Mean/std/min/max per phase
Plot execution stats	`.plot_execution_stats()`	Compare mean execution times across steps

Key takeaway: Timing analysis is hierarchical — start with step-level timings to find slow steps, then drill into execution-level timings to understand why. Step phases cover orchestration (resolve, batch, execute, commit, compact, etc.) while execution phases cover the operation lifecycle (setup, execute, record, etc.).

Next steps¶

Batching and Performance -- Tune batching parameters based on timing insights
Provenance Graphs -- Visualize the pipeline you just profiled
Execution Flow -- Understand the phases that timing analysis measures
Configuring Execution -- Resource and scheduling configuration