Contributors:
Ag Stephens (CEDA/STFC)
Carsten Ehbrecht (DKRZ)
Bouwe Andela (ESMValTool)
Rhys Evans (STFC)
https://github.com/roocs/woodpecker
Photo: Joshlaymon, Wikimedia Commons, CC BY-SA 3.0.
A lightweight Python interface for applying climate-data adaptations.
A fix can correct a known issue or adapt data to service requirements.
Different interfaces are the main obstacle: each workflow needs its own integration to reuse existing fixes.
It would be really great if we could all work together on the fixes package to avoid developing fragmented fixes solutions again.
— Bouwe Andela, ESMValTool
A thin common API for independent plugins.
FixFunction.Projects own their fixes; Woodpecker provides the common interface.
Implementation sketch — Celsius input already checked; registration and metadata omitted.
| Prefix (core package) | Suffix (from class name) |
|---|---|
woodpecker |
normalize_tas_units_to_kelvin |
ID = prefix.suffix. Updates tas.nc; add --dry-run to preview.
Already in Rook: shared plugins replace adaptations built into the service.
Where in CDS should we try this next?
Technical solution: independent fix plugins use a common API.
The hope: data producers and service providers develop, review, and maintain fix plugins together on GitHub.
First step for CDS: data producers and service providers review and test the CMIP6 Decadal adaptations together for the upcoming data update.
Many projects have developed useful repair and standardization code:
These implementations reflect different data, communities, and use cases. That specialization is useful.
The problem is the lack of a small common contract for finding, referencing, and running them.
Woodpecker does not aim to become the single fixing library for every climate-data project.
It provides:
The plugins contain most of the domain-specific work.
Plugins implement focused, testable FixFunction subclasses and may be narrow and project-specific.
Each project controls its fixes, dependencies, and scope.
Woodpecker gives each plugin the same entry point while its maintainers keep control of the implementation.
Fix
One named data adaptation exposed through a stable identifier. It may correct an issue or meet service requirements.
Recipe
An ordered set of fixes, options, matching rules, and supporting links.
Plugin
A separately owned package that provides fixes and recipes for a project, dataset family, or use case.
Interface sketch; registration and metadata omitted.
FixFunction base class.apply() method performs the repair.The stable fix ID combines two parts:
| Plugin prefix | Fix suffix | Stable fix ID |
|---|---|---|
xmip |
rename_cmip6_axes |
xmip.rename_cmip6_axes |
Woodpecker joins the prefix and suffix with a dot.
xmip.Woodpecker provides registration, discovery, execution, results, and provenance.
Excerpt from the xMIP recipe; options and other steps omitted.
The identifier is independent of the calling environment.
The same fix or recipe can be referenced by:
The portal can point to a maintained, executable definition instead of reproducing repair instructions.
The Woodpecker documentation includes an interactive overview of all registered fixes:
Example: woodpecker.normalize_tas_units_to_kelvin
This browser is a demonstration of how a portal, an Errata entry, or documentation can refer to one precise fix.
A proposed integration between issue documentation and executable repair logic:
The plugin remains the authoritative implementation.
The xMIP plugin is installed; dataset is an xarray dataset.
Both routes use the same identifiers, plugins, and recipes.
Matching, separate checks, and dry-run previews are also available when needed.
Rook is a service for remote operations on large climate datasets. It is used by the Copernicus Climate Data Store and is being considered as a processing service for ESGF-NG.
Woodpecker prepares the data. Rook operates on it.
| Plugin or example | Focus |
|---|---|
| Atlas | Production adaptations for the Copernicus Climate Data Store |
| CMIP6 Decadal | Production Decadal adaptations for the Copernicus Climate Data Store |
| CMIP6 | Dummy plugin used as a placeholder |
| ESMValTool CMIP7 example | CMIP7 and ESA-CCI examples based on the ESMValTool fixer prototype |
| xMIP | xMIP-style CMIP6 preprocessing exposed as Woodpecker fixes and recipes |
These plugins do not define the limit of Woodpecker.
Other projects can provide independent plugins while keeping their own code, scope, governance, and users.
It would be really great if we could all work together on the fixes package to avoid developing fragmented fixes solutions again.
— Bouwe Andela, ESMValTool
A common Woodpecker interface allows cooperation without forcing every project into one implementation:
The aim is for data producers and service providers to develop, review, and maintain fix plugins together on GitHub.
FixFunction subclasses?Choose one known dataset issue and let a data producer and service provider implement and test the plugin fix together through GitHub.
prefix.suffix identifier and implements the FixFunction pattern.Work together through a shared pattern while projects keep ownership of their fixes.
pip install roocs-woodpecker