RecipeCatalog example¶

Shows how a catalog combines curated recipe stores and generated auto recipes behind one lookup surface.

In [1]:

from pathlib import Path
from tempfile import TemporaryDirectory

from woodpecker.recipes import DatasetMatcher, FixRef, Recipe
from woodpecker.stores import AutoRecipeStore, JsonRecipeStore, RecipeCatalog
from woodpecker.testing import make_cmip6

from pathlib import Path from tempfile import TemporaryDirectory from woodpecker.recipes import DatasetMatcher, FixRef, Recipe from woodpecker.stores import AutoRecipeStore, JsonRecipeStore, RecipeCatalog from woodpecker.testing import make_cmip6

Create a CMIP6-like dataset and a small curated recipe store.

In [2]:

dataset = make_cmip6(overrides={"units": "degC"}, seed=7)

tmp = TemporaryDirectory()
store = JsonRecipeStore(Path(tmp.name) / "recipes.yaml")
store.save_recipe(
    Recipe(
        id="cmip6.curated_units",
        description="Curated CMIP6 units recipe",
        match=DatasetMatcher(dataset_id_patterns=["CMIP6.CMIP.*.Amon.tas.*"]),
        steps=[FixRef(id="woodpecker.normalize_tas_units_to_kelvin")],
    )
)

catalog = RecipeCatalog([store, AutoRecipeStore()])

dataset = make_cmip6(overrides={"units": "degC"}, seed=7) tmp = TemporaryDirectory() store = JsonRecipeStore(Path(tmp.name) / "recipes.yaml") store.save_recipe( Recipe( id="cmip6.curated_units", description="Curated CMIP6 units recipe", match=DatasetMatcher(dataset_id_patterns=["CMIP6.CMIP.*.Amon.tas.*"]), steps=[FixRef(id="woodpecker.normalize_tas_units_to_kelvin")], ) ) catalog = RecipeCatalog([store, AutoRecipeStore()])

The catalog lists recipes from all sources and deduplicates by recipe id.

In [3]:

[recipe.id for recipe in catalog.list_recipes()[:8]]

[recipe.id for recipe in catalog.list_recipes()[:8]]

Out[3]:

['cmip6.curated_units',
 'cmip6_decadal.time_metadata',
 'cmip6_decadal.calendar_normalization',
 'cmip6_decadal.realization_variable',
 'cmip6_decadal.coordinates_encoding_cleanup',
 'cmip6_decadal.realization_comment_normalization',
 'cmip6_decadal.realization_dtype_normalization',
 'cmip6_decadal.fillvalue_encoding_cleanup']

Lookup returns the curated recipe first, followed by the generated auto recipe.

In [4]:

matched_plans = catalog.lookup(dataset)
[recipe.id for recipe in matched_plans]

matched_plans = catalog.lookup(dataset) [recipe.id for recipe in matched_plans]

Out[4]:

['cmip6.curated_units',
 'woodpecker.normalize_tas_units_to_kelvin',
 'xmip.broadcast_lon_lat',
 'xmip.rename_cmip6_axes']

The same catalog resolves recipe ids and aliases.

In [5]:

catalog.get_recipe("cmip6.curated_units")

catalog.get_recipe("cmip6.curated_units")

Out[5]:

Recipe(id='cmip6.curated_units', aliases=[], description='Curated CMIP6 units recipe', match=DatasetMatcher(attrs={}, dataset_id_patterns=['CMIP6.CMIP.*.Amon.tas.*'], path_patterns=[]), steps=[FixRef(id='woodpecker.normalize_tas_units_to_kelvin', phase='apply', options={}, links=[])], links=[])

In [6]:

tmp.cleanup()

tmp.cleanup()