Metrics and Filtering

What you’ll learn¶

Filter artifacts by metric values using a single metric source
Combine criteria from multiple metric sources into one filter
Inspect metric values to understand what passed and why

Prerequisites: First Pipeline Estimated time: 10 minutes

Quality gating is one of the most common pipeline patterns: generate a batch of artifacts, compute quality metrics, and keep only those that meet your criteria. Filter does this — it routes existing artifacts based on metric values without creating new ones.

Section	What you’ll do
Single-metric filter	Filter by one criterion with auto-discovered metrics
Multi-metric filter	Combine criteria from multiple metric sources with AND semantics

from __future__ import annotations

from artisan.operations.curator import Filter
from artisan.operations.examples import (
    DataGeneratorWithMetrics,
    DataTransformer,
    MetricCalculator,
)
from artisan.orchestration import PipelineManager
from artisan.utils import tutorial_setup
from artisan.visualization import (
    build_macro_graph,
    build_micro_graph,
    inspect_metrics,
)

Graph legend: See Sources and Sequencing for box/arrow key.

Single-metric filter¶

DataGeneratorWithMetrics produces datasets and associated metrics in one step. Each dataset gets quality scores like mean_score and std_score. Filter auto-discovers these metrics through provenance — you provide the passthrough input (the artifacts to filter) and the criteria.

When to use: Quality gating on metrics that were co-produced with the artifacts being filtered.

env_single = tutorial_setup("single_metric_filter")


def run_single_metric():
    pipeline = PipelineManager.create(
        name="single_metric_filter",
        delta_root=env_single.delta_root,
        staging_root=env_single.staging_root,
        working_root=env_single.working_root,
    )
    output = pipeline.output

    # Generate 5 datasets with co-produced quality metrics
    pipeline.run(
        operation=DataGeneratorWithMetrics,
        name="generate",
        params={"count": 5, "seed": 42},
    )

    # Keep only datasets where mean_score > 0.5
    pipeline.run(
        operation=Filter,
        name="filter",
        inputs={"passthrough": output("generate", "datasets")},
        params={
            "criteria": [
                {"metric": "mean_score", "operator": "gt", "value": 0.5},
            ]
        },
    )

    # Transform the datasets that passed
    pipeline.run(
        operation=DataTransformer,
        name="transform",
        inputs={"dataset": output("filter", "passthrough")},
        params={"seed": 100},
    )
    return pipeline.finalize()


result_single = run_single_metric()

build_macro_graph(env_single.delta_root)

build_micro_graph(env_single.delta_root)

Inspecting the metrics¶

inspect_metrics reads metric artifacts from the Delta Lake store and displays them as a table. The output below shows what the generate step produced — these are the values Filter evaluated.

inspect_metrics(env_single.delta_root, step_number=0)

Each row is one dataset. The mean_score column is what our criterion checked: datasets with mean_score > 0.5 passed through to the transform step; the rest were dropped. Filter doesn’t delete anything — dropped artifacts remain in the store, they are not routed to downstream steps.

A few things to note about how this worked:

Auto-discovery: Filter discovers metrics via forward provenance walk from the passthrough artifacts. Because DataGeneratorWithMetrics co-produces datasets and metrics with output-to-output lineage, Filter resolves which metric belongs to which dataset automatically.
Bare field names: Criteria always use bare field names like "mean_score". When field names collide across metric sources, add step or step_number to the criterion to disambiguate.
Passthrough output: Filter’s output role is also named "passthrough". The artifacts that come out are the same artifact IDs that went in.

Multi-metric filter¶

When metrics come from different steps, Filter still auto-discovers them via forward provenance walk — as long as the field names don’t collide across sources, you use bare field names just like the single-source case.

Here, DataGeneratorWithMetrics produces central-tendency stats (mean_score, std_score), and MetricCalculator computes distribution stats (distribution.min, distribution.range, summary.cv). Because all field names are unique across the two sources, Filter resolves them automatically.

When to use: Quality gating that requires metrics from multiple independent sources (e.g., mean_score > 0.3 AND distribution.range < 0.8).

env_multi = tutorial_setup("multi_metric_filter")


def run_multi_metric():
    pipeline = PipelineManager.create(
        name="multi_metric_filter",
        delta_root=env_multi.delta_root,
        staging_root=env_multi.staging_root,
        working_root=env_multi.working_root,
    )
    output = pipeline.output

    # Generate datasets + central-tendency metrics (mean, std)
    pipeline.run(
        operation=DataGeneratorWithMetrics,
        name="generate",
        params={"count": 5, "seed": 42},
    )

    # Compute distribution metrics (min, max, median, range, CV)
    pipeline.run(
        operation=MetricCalculator,
        name="score",
        inputs={"dataset": output("generate", "datasets")},
    )

    # Filter on criteria from BOTH metric streams
    # Forward provenance walk discovers metrics from both sources automatically.
    # Field names are unique across sources, so no disambiguation needed.
    pipeline.run(
        operation=Filter,
        name="filter",
        inputs={"passthrough": output("generate", "datasets")},
        params={
            "criteria": [
                {"metric": "mean_score", "operator": "gt", "value": 0.3},
                {
                    "metric": "distribution.range",
                    "operator": "lt",
                    "value": 0.9,
                },
            ]
        },
    )

    # Transform the datasets that passed both criteria
    pipeline.run(
        operation=DataTransformer,
        name="transform",
        inputs={"dataset": output("filter", "passthrough")},
        params={"seed": 200},
    )
    return pipeline.finalize()


result_multi = run_multi_metric()

inspect_metrics(env_multi.delta_root, step_number=1)

build_macro_graph(env_multi.delta_root)

build_micro_graph(env_multi.delta_root)

Inspecting multi-metric results¶

With two metric sources, viewing all metrics across the pipeline shows both central-tendency and distribution stats side by side.

inspect_metrics(env_multi.delta_root)

Filter applies AND semantics: an artifact must satisfy every criterion to pass. Here, "mean_score" evaluates against the co-produced metrics from the generate step, and "distribution.range" evaluates against the distribution metrics computed by the score step. Filter discovers both metric sources via forward provenance walk from the passthrough artifacts and uses lineage matching to pair each passthrough artifact with its corresponding metrics from both streams.

Because the field names are unique across sources (mean_score only comes from generate, distribution.range only comes from score), no disambiguation is needed. When field names do collide across sources, add step or step_number to the criterion:

{"metric": "score", "operator": "gt", "value": 0.8, "step": "calc_quality"}

Summary¶

In this tutorial you learned two filtering patterns:

Single-metric filter — filter by one criterion with auto-discovered metrics via provenance (bare field names, no explicit metric wiring)
Multi-metric filter — combine criteria from multiple metric sources with AND semantics; Filter auto-discovers all descendant metrics via forward provenance walk

Key concepts:

Filter has passthrough semantics — it routes existing artifacts without creating new ones
Criteria always use bare field names ("mean_score", "distribution.range") — Filter discovers metrics automatically
When field names collide across metric sources, add step or step_number to the criterion to disambiguate
Nested metrics use dot-path notation ("distribution.range")
Use inspect_metrics to view per-artifact metric values across the pipeline

Next steps¶

Multi-Input Operations — operations that consume multiple input roles
Operations Model — how creators and curators differ under the hood
Provenance System — the forward provenance walk that powers metric discovery