Core#

wetterdienst.core.interpolate#

Interpolation for weather data.

Module Contents#

Functions#

get_interpolated_df

Get the interpolated DataFrame for the given request and location.

request_stations

Request the stations for the interpolation.

apply_station_values_per_parameter

Apply the station values to the parameter data.

calculate_interpolation

Calculate the interpolation for the given data.

get_valid_station_groups

Get all valid station groups that cover the given point.

get_station_group_ids

Get the station group ids that are a subset of the given values.

apply_interpolation

Apply interpolation to a row of data.

Data#

log

API#

wetterdienst.core.interpolate.log

‘getLogger(…)’

wetterdienst.core.interpolate.get_interpolated_df(request: wetterdienst.model.request.TimeseriesRequest, latitude: float, longitude: float) polars.DataFrame

Get the interpolated DataFrame for the given request and location.

wetterdienst.core.interpolate.request_stations(request: wetterdienst.model.request.TimeseriesRequest, latitude: float, longitude: float, utm_x: float, utm_y: float) tuple[dict, dict]

Request the stations for the interpolation.

Args: request: TimeseriesRequest object latitude: latitude of the point to interpolate longitude: longitude of the point to interpolate utm_x: longitude in UTM of the point to interpolate utm_y: latitude in UTM of the point to interpolate

Returns: tuple containing the stations dict and the parameter dict

wetterdienst.core.interpolate.apply_station_values_per_parameter(result_df: polars.DataFrame, stations_ranked: wetterdienst.model.result.StationsResult, param_dict: dict, station: dict, *, valid_station_groups_exists: bool) None

Apply the station values to the parameter data.

Args: result_df: DataFrame containing the station values stations_ranked: stations_result with stations ranked by distance param_dict: dict containing the parameter data station: dict containing the station data min_gain_of_value_pairs: minimum gain of value pairs to add a station num_additional_stations: number of additional stations to add if the gain is not reached valid_station_groups_exists: bool indicating if valid station groups exist

Returns: None - the parameter data is updated in place

wetterdienst.core.interpolate.calculate_interpolation(utm_x: float, utm_y: float, stations_dict: dict, param_dict: dict, use_nearby_station_distance: float | None) polars.DataFrame

Calculate the interpolation for the given data.

Args: utm_x: longitude in UTM utm_y: latitude in UTM stations_dict: dict containing the station data including the location param_dict: dict containing the parameter data use_nearby_station_distance: distance in km to use nearby stations for interpolation

Returns: DataFrame containing the interpolated data

wetterdienst.core.interpolate.get_valid_station_groups(stations_dict: dict, utm_x: float, utm_y: float) queue.Queue

Get all valid station groups that cover the given point.

Args: stations_dict: dict containing the station data including the location utm_x: longitude in UTM utm_y: latitude in UTM

Returns: Queue containing the valid station groups

wetterdienst.core.interpolate.get_station_group_ids(valid_station_groups: queue.Queue, vals_index: frozenset) list

Get the station group ids that are a subset of the given values.

wetterdienst.core.interpolate.apply_interpolation(row: dict, stations_dict: dict, valid_station_groups: queue.Queue, resolution: str, dataset: str, parameter: str, utm_x: float, utm_y: float, nearby_stations: list[str]) tuple[str, str, str, float | None, float | None, list[str]]

Apply interpolation to a row of data.

Args: row: dict containing the data across collected stations for a specific date stations_dict: dict containing the station data including the location valid_station_groups: Queue containing the valid station groups to use for interpolation resolution: resolution name dataset: dataset name parameter: parameter name utm_x: longitude in UTM utm_y: latitude in UTM nearby_stations: list of nearby stations

Returns: tuple containing the resolution name, dataset name, parameter name, interpolated value, mean distance of the stations used for interpolation and the station ids used for interpolation

wetterdienst.model.request#

Core for timeseries information of a source.

Module Contents#

Classes#

TimeseriesRequest

Core class for timeseries information of a source.

Data#

log

EARTH_RADIUS_KM

_PARAMETER_TYPE_SINGULAR

_PARAMETER_TYPE

_DATETIME_TYPE

API#

wetterdienst.model.request.log

‘getLogger(…)’

wetterdienst.model.request.EARTH_RADIUS_KM

6371

wetterdienst.model.request._PARAMETER_TYPE_SINGULAR

None

wetterdienst.model.request._PARAMETER_TYPE

None

wetterdienst.model.request._DATETIME_TYPE

None

class wetterdienst.model.request.TimeseriesRequest

Core class for timeseries information of a source.

metadata: wetterdienst.model.metadata.MetadataModel

‘field(…)’

_values: wetterdienst.model.values.TimeseriesValues

‘field(…)’

_history: wetterdienst.model.history.TimeseriesHistory

‘field(…)’

parameters: wetterdienst.model.request._PARAMETER_TYPE

None

start_date: wetterdienst.model.request._DATETIME_TYPE

None

end_date: wetterdienst.model.request._DATETIME_TYPE

None

settings: wetterdienst.settings.Settings | dict

‘field(…)’

__post_init__() None

Post init method to validate the settings and convert the timestamps.

_base_columns: ClassVar

(‘resolution’, ‘dataset’, ‘station_id’, ‘start_date’, ‘end_date’, ‘latitude’, ‘longitude’, ‘height’,…

interpolatable_parameters: ClassVar

None

static _parse_station_id(series: polars.Series) polars.Series

Parse station_id column to string.

Args: series: Series containing station ids.

Returns: pl.Series: Series with station ids as strings.

static convert_timestamps(start_date: wetterdienst.model.request._DATETIME_TYPE, end_date: wetterdienst.model.request._DATETIME_TYPE) tuple[None, None] | tuple[datetime.datetime, datetime.datetime]

Convert timestamps to datetime objects.

Args: start_date: Start date of the request. end_date: End date of the request.

Returns: tuple[None, None] | tuple[dt.datetime, dt.datetime]: Start and end date of the request.

classmethod is_configured() bool

Return True if this provider’s required credentials are present (env var / settings).

This is a cheap, offline check. Override in subclasses that require authentication.

classmethod is_valid(settings: wetterdienst.settings.Settings | None = None) bool

Return True if the provider’s credentials are valid (authenticated successfully).

This may perform a lightweight network probe and should cache the result. Only called when is_configured() is True. Override in auth-requiring subclasses. The optional settings argument lets callers pass a custom Settings instance; implementations that ignore it fall back to Settings() internally.

classmethod discover(resolutions: str | wetterdienst.metadata.resolution.Resolution | wetterdienst.model.metadata.ResolutionModel | collections.abc.Sequence[str | wetterdienst.metadata.resolution.Resolution | wetterdienst.model.metadata.ResolutionModel] | None = None, datasets: str | wetterdienst.model.metadata.DatasetModel | collections.abc.Sequence[str | wetterdienst.model.metadata.DatasetModel] | None = None) dict

Discover metadata for the given resolutions and datasets.

Each level carries its own description, so the shape has a place for one::

{resolution: {"description": ..., "datasets": {dataset: {"description": ...,
 "parameters": [{"name": ..., "name_original": ..., "unit_type": ..., "unit": ...,
 "description": ...}]}}}}

Args: resolutions: Resolutions to discover metadata for. datasets: Datasets to discover metadata for.

Returns: dict: Metadata for the given resolutions and datasets.

static _coerce_meta_fields(df: polars.DataFrame) polars.DataFrame

Coerce metadata fields to the correct types.

abstractmethod _all() polars.LazyFrame

Implement this method to get all stations.

Returns: pl.LazyFrame: All stations.

all() wetterdienst.model.result.StationsResult

Get all stations.

Returns: StationsResult: All stations.

filter_by_station_id(station_id: str | tuple[str, ...] | list[str]) wetterdienst.model.result.StationsResult

Filter stations by station_id.

Args: station_id: Station id or list of station ids.

Returns: StationsResult: Filtered stations.

filter_by_name(name: str, rank: int = 1, threshold: float = 0.8) wetterdienst.model.result.StationsResult

Filter stations by name.

Args: name: Name of the station. rank: Maximum number of matches to return, best score first (default 1). threshold: Threshold for the fuzzy search.

Returns: StationsResult: Filtered stations.

filter_by_rank(latlon: tuple[float, float], rank: int) wetterdienst.model.result.StationsResult

Filter stations by rank.

Rank is defined by distance to the requested point. The resulting StationsResult.df holds all stations sorted by distance, not just rank rows: because we cannot know upfront which stations actually carry data for the request, the rank limit is applied lazily while collecting values. Value collection walks the distance-sorted stations and stops once rank stations with data (per the ts_skip_empty / ts_skip_threshold / ts_skip_criteria settings) have been consumed. The stations that ended up contributing values are then exposed via ValuesResult.df_stations.

In other words, use stations.values.all().df_stations (not stations.df) to see the rank closest stations that actually returned data. Set ts_skip_empty=False to simply take the rank closest stations regardless of data availability.

Args: latlon: Latitude and longitude for the requested point. rank: Number of stations requested.

Returns: StationsResult: Stations sorted by distance (see note above on rank).

filter_by_distance(latlon: tuple[float, float], distance: float, unit: str = 'km') wetterdienst.model.result.StationsResult

Filter stations by distance.

Args: latlon: Latitude and longitude for the requested point. distance: Maximum distance to the requested point. unit: Unit of the distance.

Returns: StationsResult: Filtered stations.

filter_by_bbox(left: float, bottom: float, right: float, top: float) wetterdienst.model.result.StationsResult

Filter stations by bounding box.

Args: left: Left border of the bounding box. bottom: Bottom border of the bounding box. right: Right border of the bounding box. top: Top border of the bounding box.

Returns: StationsResult: Filtered stations.

filter_by_sql(sql: str) wetterdienst.model.result.StationsResult

Filter stations by SQL query.

Args: sql: SQL query to filter stations by.

Returns: StationsResult: Filtered stations.

interpolate(latlon: tuple[float, float]) wetterdienst.model.result.InterpolatedValuesResult

Interpolate values across multiple stations.

Interpolation means we interpolate the values of the closest available stations to the requested point.

Args: latlon: Latitude and longitude for the requested point.

Returns: InterpolatedValuesResult: Interpolated values.

interpolate_by_station_id(station_id: str) wetterdienst.model.result.InterpolatedValuesResult

Use .interpolate with station_id instead of latlon.

summarize(latlon: tuple[float, float]) wetterdienst.model.result.SummarizedValuesResult

Summarize values across multiple stations.

Summarize means we take any available data of the closest station as representative for the timestamp.

summarize_by_station_id(station_id: str) wetterdienst.model.result.SummarizedValuesResult

Use .summarize with station_id instead of latlon.

_get_latlon_by_station_id(station_id: str) tuple[float, float]

Get latlon for a station_id.

Used for .summary/.interpolate. Typically, we expect a latlon tuple of floats, but we want users to be able to request for a station id as well.

static _create_station_id_from_string(string: str) str

Create station id from string.

Used for interpolation and summarization data

wetterdienst.model.values#

Core for sources of timeseries where data is related to a station.

Module Contents#

Classes#

TimeseriesValues

Core for sources of timeseries where data is related to a station.

Data#

log

API#

wetterdienst.model.values.log

‘getLogger(…)’

class wetterdienst.model.values.TimeseriesValues

Bases: abc.ABC

Core for sources of timeseries where data is related to a station.

sr: wetterdienst.model.result.StationsResult

None

stations_counter: int

0

stations_collected: list[str]

‘field(…)’

unit_converter: wetterdienst.model.unit.UnitConverter

‘field(…)’

_date_fields: ClassVar

[‘date’, ‘start_date’, ‘end_date’]

__post_init__() None

Post-initialization of the TimeseriesValues object.

classmethod from_stations(stations: wetterdienst.model.result.StationsResult) wetterdienst.model.values.TimeseriesValues

Create a new instance of the class from a StationsResult object.

property _meta_fields: dict[str, Any]

Get metadata fields for the DataFrame.

property timezone_data: str

Get timezone data for the station.

_adjust_start_end_date(start_date: datetime.datetime, end_date: datetime.datetime, tzinfo: zoneinfo.ZoneInfo, resolution: wetterdienst.metadata.resolution.Resolution) tuple[datetime.datetime, datetime.datetime]

Adjust start and end date for a given resolution.

_get_complete_dates(start_date: datetime.datetime, end_date: datetime.datetime, resolution: wetterdienst.metadata.resolution.Resolution) polars.Series

Get a complete date range for a given start and end date and resolution.

_get_timezone_from_station(station_id: str) str

Get timezone information for explicit station.

This is used to set the correct timezone for the timestamps of the returned values.

_get_base_df(start_date: datetime.datetime, end_date: datetime.datetime, resolution: wetterdienst.metadata.resolution.Resolution) polars.DataFrame

Create a base DataFrame with all dates for a given station.

_convert_units(df: polars.DataFrame, dataset: wetterdienst.model.metadata.DatasetModel) polars.DataFrame

Convert values to metric units with help of conversion factors.

_create_conversion_lambdas(dataset: wetterdienst.model.metadata.DatasetModel) dict[str, collections.abc.Callable[[Any], Any]]

Create conversion factors based on a given dataset.

_build_complete_df(df: polars.DataFrame, station_id: str, resolution: wetterdienst.metadata.resolution.Resolution) polars.DataFrame

Build a complete DataFrame with all dates for a given station.

_organize_df_columns(df: polars.DataFrame, station_id: str, dataset: wetterdienst.model.metadata.DatasetModel) polars.DataFrame

Reorder columns in DataFrame to match the expected order of columns.

_meta_enum_columns: ClassVar

(‘station_id’, ‘resolution’, ‘dataset’, ‘parameter’)

classmethod _cast_metadata_to_enum(df: polars.DataFrame) polars.DataFrame

Cast the low-cardinality metadata columns to Enum to reduce the memory footprint.

The categories are taken from the values actually present in the frame (not from the request metadata), so the cast never fails on provider-specific casing or humanization quirks (e.g. WSV emitting w while the metadata declares W). These columns repeat every row, so integer-backed Enum codes roughly halve the size of tidy value frames.

query() collections.abc.Iterator[wetterdienst.model.result.ValuesResult]

Query data for all stations and parameters and return a DataFrame for each station.

_get_available_datasets(df: polars.DataFrame) list[wetterdienst.model.metadata.DatasetModel]

Extract available datasets for the station.

_collect_station_data(station_id: str, available_datasets: list[wetterdienst.model.metadata.DatasetModel]) polars.DataFrame

Collect and process data for a single station.

_process_dataset(station_id: str, dataset: wetterdienst.model.metadata.DatasetModel, parameters: collections.abc.Iterator[wetterdienst.model.metadata.ParameterModel]) polars.DataFrame

Process data for a specific dataset.

abstractmethod _collect_station_parameter_or_dataset(station_id: str, parameter_or_dataset: wetterdienst.model.metadata.ParameterModel | wetterdienst.model.metadata.DatasetModel) polars.DataFrame

Collect data for a station and a single parameter or dataset.

_widen_df(df: polars.DataFrame) polars.DataFrame

Widen a dataframe with each row having one timestamp, parameter, value and quality.

Example: date parameter value quality 1971-01-01 precipitation_height 0 0 1971-01-01 temperature_air_mean_2m 10 0

becomes

date precipitation_height qn_precipitation_height 1971-01-01 0 0 temperature_air_mean_2m … 10 …

Args: df: DataFrame with columns date, parameter, value and quality.

Returns: DataFrame with columns date, parameter, value and quality as columns.

all() wetterdienst.model.result.ValuesResult

Collect all data for all stations and parameters and return a single DataFrame.

to_target(target: str, if_exists: Literal[replace, append, fail, skip] = 'fail') None

Wrap to_target of all queried results.

static _humanize(df: polars.DataFrame, humanized_parameters_mapping: dict[str, str]) polars.DataFrame

Humanize parameter names in a DataFrame.

_create_humanized_parameters_mapping() dict[str, str]

Create mapping of original to humanized parameter names.

_get_actual_percentage(df: polars.DataFrame) float

Get the percentage of actual values in the DataFrame.

This is used to skip stations with too many missing values.

wetterdienst.model.result#

Result classes for timeseries data.

Module Contents#

Classes#

StationsFilter

Enumeration for stations filter.

_Provider

Type definition for provider metadata.

_Producer

Type definition for producer metadata.

_Metadata

Type definition for metadata.

_Station

Type definition for station.

_StationsDict

Type definition for dictionary of stations.

_OgcFeatureProperties

Type definition for OGC feature properties.

_OgcFeatureGeometry

Type definition for OGC feature geometry.

_StationsOgcFeature

Type definition for OGC feature of stations.

_StationsOgcFeatureCollectionData

Type definition for OGC feature collection data of stations.

_StationsOgcFeatureCollection

Type definition for OGC feature collection of stations.

StationsResult

Result class for stations.

_ValuesItemDict

Type definition for dictionary of values.

_ValuesDict

Type definition for dictionary of values.

_ValuesResult

Result class for values.

_ValuesOgcFeature

Type definition for OGC feature of values.

_ValuesOgcFeatureCollectionData

Type definition for OGC feature collection data of values.

_ValuesOgcFeatureCollection

Type definition for OGC feature collection of values.

ValuesResult

Result class for values.

HistoryResult

Result class for history data.

_InterpolatedOrSummarizedOgcFeatureProperties

Type definition for OGC feature properties of interpolated or summarized values.

_InterpolatedValuesItemDict

Type definition for dictionary of interpolated values.

_InterpolatedValuesDict

Type definition for dictionary of interpolated values.

_InterpolatedValuesOgcFeature

Type definition for OGC feature of interpolated values.

_InterpolatedValuesOgcFeatureCollectionData

Type definition for OGC feature collection data of interpolated values.

_InterpolatedValuesOgcFeatureCollection

Type definition for OGC feature collection of interpolated values.

InterpolatedValuesResult

Result class for interpolated values.

_SummarizedValuesItemDict

Format summarized values as dictionary.

_SummarizedValuesDict

Format summarized values as dictionary.

_SummarizedValuesOgcFeature

Format summarized values as OGC feature.

_SummarizedValuesOgcFeatureCollectionData

Format summarized values as OGC feature collection data.

_SummarizedValuesOgcFeatureCollection

Format summarized values as OGC feature collection.

SummarizedValuesResult

Calculate summary of stations and parameters.

API#

class wetterdienst.model.result.StationsFilter

Bases: enum.Enum

Enumeration for stations filter.

This should help determine why only a subset of stations was returned.

ALL

‘all’

BY_STATION_ID

‘by_station_id’

BY_NAME

‘by_name’

BY_RANK

‘by_rank’

BY_DISTANCE

‘by_distance’

BY_BBOX

‘by_bbox’

BY_SQL

‘by_sql’

class wetterdienst.model.result._Provider

Bases: typing_extensions.TypedDict

Type definition for provider metadata.

Initialization

Initialize self. See help(type(self)) for accurate signature.

name_local: str

None

name_english: str

None

country: str

None

copyright: str

None

url: str

None

class wetterdienst.model.result._Producer

Bases: typing_extensions.TypedDict

Type definition for producer metadata.

Initialization

Initialize self. See help(type(self)) for accurate signature.

name: str

None

version: str

None

repository: str

None

documentation: str

None

doi: str

None

class wetterdienst.model.result._Metadata

Bases: typing_extensions.TypedDict

Type definition for metadata.

Initialization

Initialize self. See help(type(self)) for accurate signature.

provider: wetterdienst.model.result._Provider

None

producer: wetterdienst.model.result._Producer

None

class wetterdienst.model.result._Station

Bases: typing_extensions.TypedDict

Type definition for station.

Initialization

Initialize self. See help(type(self)) for accurate signature.

resolution: str

None

dataset: str

None

station_id: str

None

start_date: str | None

None

end_date: str | None

None

latitude: float

None

longitude: float

None

height: float

None

name: str

None

state: str | None

None

class wetterdienst.model.result._StationsDict

Bases: typing_extensions.TypedDict

Type definition for dictionary of stations.

Initialization

Initialize self. See help(type(self)) for accurate signature.

metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]

None

stations: list[wetterdienst.model.result._Station]

None

class wetterdienst.model.result._OgcFeatureProperties

Bases: typing_extensions.TypedDict

Type definition for OGC feature properties.

Initialization

Initialize self. See help(type(self)) for accurate signature.

resolution: str

None

dataset: str

None

id: str

None

name: str

None

state: str | None

None

start_date: str | None

None

end_date: str | None

None

class wetterdienst.model.result._OgcFeatureGeometry

Bases: typing_extensions.TypedDict

Type definition for OGC feature geometry.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[Point]

None

coordinates: list[float]

None

class wetterdienst.model.result._StationsOgcFeature

Bases: typing_extensions.TypedDict

Type definition for OGC feature of stations.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[Feature]

None

properties: wetterdienst.model.result._OgcFeatureProperties

None

geometry: wetterdienst.model.result._OgcFeatureGeometry

None

class wetterdienst.model.result._StationsOgcFeatureCollectionData

Bases: typing_extensions.TypedDict

Type definition for OGC feature collection data of stations.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[FeatureCollection]

None

features: list[wetterdienst.model.result._StationsOgcFeature]

None

class wetterdienst.model.result._StationsOgcFeatureCollection

Bases: typing_extensions.TypedDict

Type definition for OGC feature collection of stations.

Initialization

Initialize self. See help(type(self)) for accurate signature.

metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]

None

data: wetterdienst.model.result._StationsOgcFeatureCollectionData

None

class wetterdienst.model.result.StationsResult

Bases: wetterdienst.io.export.ExportMixin

Result class for stations.

stations: wetterdienst.model.request.TimeseriesRequest | wetterdienst.provider.dwd.mosmix.DwdMosmixRequest | wetterdienst.provider.dwd.dmo.DwdDmoRequest

None

df: polars.DataFrame

None

df_all: polars.DataFrame

None

stations_filter: wetterdienst.model.result.StationsFilter

None

rank: int | None

None

property settings: wetterdienst.Settings

Get settings for the request.

property parameters: list[wetterdienst.model.metadata.ParameterModel]

Get parameters from the request.

property values: wetterdienst.model.values.TimeseriesValues

Get values from the request.

property history: wetterdienst.model.history.TimeseriesHistory

Get history from the request.

property start_date: datetime.datetime | None

Get start date from the request.

property end_date: datetime.datetime | None

Get end date from the request.

property station_id: polars.Series

Get station IDs from the DataFrame.

get_metadata() wetterdienst.model.result._Metadata

Get metadata for the provider and producer.

to_dict(*, with_metadata: bool = False) wetterdienst.model.result._StationsDict

Format station information as dictionary.

Args: with_metadata: bool whether to include metadata

Returns: Dictionary with station information.

to_json(*, with_metadata: bool = False, indent: int | bool | None = 4) str

Format station information as JSON.

Args: with_metadata: bool whether to include metadata indent: int or bool whether to indent the JSON

Returns: JSON string with station information.

to_ogc_feature_collection(*, with_metadata: bool = False, **_kwargs) wetterdienst.model.result._StationsOgcFeatureCollection

Format station information as OGC feature collection.

Will be used by .to_geojson().

Args: with_metadata: bool whether to include metadata (information about the provider and producer)

Returns: Dictionary with station information as OGC feature collection.

to_plot(**_kwargs: dict) plotly.graph_objects.Figure

Create a plotly figure from the stations DataFrame.

_to_image(fmt: Literal[html, png, jpg, webp, svg, pdf], width: int | None = None, height: int | None = None, scale: float | None = None, **kwargs: dict) bytes | str

Create an image from the plotly figure.

This method is used by .to_image() to create an image for stations from the plotly figure.

class wetterdienst.model.result._ValuesItemDict

Bases: typing_extensions.TypedDict

Type definition for dictionary of values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

station_id: str

None

resolution: str

None

dataset: str

None

parameter: str

None

date: str

None

value: float | None

None

quality: float | None

None

class wetterdienst.model.result._ValuesDict

Bases: typing_extensions.TypedDict

Type definition for dictionary of values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]

None

stations: typing_extensions.NotRequired[list[wetterdienst.model.result._Station]]

None

values: list[wetterdienst.model.result._ValuesItemDict]

None

class wetterdienst.model.result._ValuesResult

Bases: wetterdienst.io.export.ExportMixin

Result class for values.

stations: wetterdienst.model.result.StationsResult

None

df: polars.DataFrame

None

static _to_dict(df: polars.DataFrame) list[wetterdienst.model.result._ValuesItemDict]

Format values as dictionary.

This method is used both by to_dict(), and to_ogc_feature_collection(), however, the latter one splits the DataFrame into multiple DataFrames by station and calls this method for each of them.

to_dict(*, with_metadata: bool = False, with_stations: bool = False) wetterdienst.model.result._ValuesDict

Format values as dictionary.

to_json(*, with_metadata: bool = False, with_stations: bool = False, indent: int | bool | None = 4) str

Format values as JSON.

filter_by_date(date: str) polars.DataFrame

Filter values by date and return a new DataFrame.

class wetterdienst.model.result._ValuesOgcFeature

Bases: typing_extensions.TypedDict

Type definition for OGC feature of values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[Feature]

None

properties: wetterdienst.model.result._OgcFeatureProperties

None

geometry: wetterdienst.model.result._OgcFeatureGeometry

None

values: list[wetterdienst.model.result._ValuesItemDict]

None

class wetterdienst.model.result._ValuesOgcFeatureCollectionData

Bases: typing_extensions.TypedDict

Type definition for OGC feature collection data of values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[FeatureCollection]

None

features: list[wetterdienst.model.result._ValuesOgcFeature]

None

class wetterdienst.model.result._ValuesOgcFeatureCollection

Bases: typing_extensions.TypedDict

Type definition for OGC feature collection of values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]

None

data: wetterdienst.model.result._ValuesOgcFeatureCollectionData

None

class wetterdienst.model.result.ValuesResult

Bases: wetterdienst.model.result._ValuesResult

Result class for values.

stations: wetterdienst.model.result.StationsResult

None

values: wetterdienst.model.values.TimeseriesValues

None

df: polars.DataFrame

None

property df_stations: polars.DataFrame

Get DataFrame with stations.

to_ogc_feature_collection(*, with_metadata: bool = False, **_kwargs) wetterdienst.model.result._ValuesOgcFeatureCollection

Format values as OGC feature collection.

to_plot(**_kwargs: dict) plotly.graph_objects.Figure

Create a plotly figure from the values DataFrame.

_to_image(fmt: Literal[html, png, jpg, webp, svg, pdf], width: int | None = None, height: int | None = None, scale: float | None = None, **kwargs: dict) bytes | str

Create an image from the plotly figure.

This method is used by .to_image() to create an image for values from the plotly figure.

class wetterdienst.model.result.HistoryResult

Result class for history data.

stations: wetterdienst.model.result.StationsResult

None

history: wetterdienst.model.history.History

None

class wetterdienst.model.result._InterpolatedOrSummarizedOgcFeatureProperties

Bases: typing_extensions.TypedDict

Type definition for OGC feature properties of interpolated or summarized values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

id: str

None

name: str

None

class wetterdienst.model.result._InterpolatedValuesItemDict

Bases: typing_extensions.TypedDict

Type definition for dictionary of interpolated values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

station_id: str

None

resolution: str

None

dataset: str

None

parameter: str

None

date: str

None

value: float | None

None

distance_mean: float | None

None

taken_station_ids: list[str]

None

class wetterdienst.model.result._InterpolatedValuesDict

Bases: typing_extensions.TypedDict

Type definition for dictionary of interpolated values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]

None

stations: typing_extensions.NotRequired[list[wetterdienst.model.result._Station]]

None

values: list[wetterdienst.model.result._InterpolatedValuesItemDict]

None

class wetterdienst.model.result._InterpolatedValuesOgcFeature

Bases: typing_extensions.TypedDict

Type definition for OGC feature of interpolated values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[Feature]

None

properties: wetterdienst.model.result._InterpolatedOrSummarizedOgcFeatureProperties

None

geometry: wetterdienst.model.result._OgcFeatureGeometry

None

stations: list[wetterdienst.model.result._Station]

None

values: list[wetterdienst.model.result._InterpolatedValuesItemDict]

None

class wetterdienst.model.result._InterpolatedValuesOgcFeatureCollectionData

Bases: typing_extensions.TypedDict

Type definition for OGC feature collection data of interpolated values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[FeatureCollection]

None

features: list[wetterdienst.model.result._InterpolatedValuesOgcFeature]

None

class wetterdienst.model.result._InterpolatedValuesOgcFeatureCollection

Bases: typing_extensions.TypedDict

Type definition for OGC feature collection of interpolated values.

Initialization

Initialize self. See help(type(self)) for accurate signature.

metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]

None

data: wetterdienst.model.result._InterpolatedValuesOgcFeatureCollectionData

None

class wetterdienst.model.result.InterpolatedValuesResult

Bases: wetterdienst.model.result._ValuesResult

Result class for interpolated values.

stations: wetterdienst.model.result.StationsResult

None

df: polars.DataFrame

None

latlon: tuple[float, float]

None

to_ogc_feature_collection(*, with_metadata: bool = False, **_kwargs) wetterdienst.model.result._InterpolatedValuesOgcFeatureCollection

Format interpolated values as OGC feature collection.

to_plot(**_kwargs: dict) plotly.graph_objects.Figure

Create a plotly figure from the values DataFrame.

_to_image(fmt: Literal[html, png, jpg, webp, svg, pdf], width: int | None = None, height: int | None = None, scale: float | None = None, **kwargs: dict) bytes | str

Create an image from the plotly figure.

This method is used by .to_image() to create an image for interpolated values from the plotly figure.

class wetterdienst.model.result._SummarizedValuesItemDict

Bases: typing_extensions.TypedDict

Format summarized values as dictionary.

Initialization

Initialize self. See help(type(self)) for accurate signature.

station_id: str

None

resolution: str

None

dataset: str

None

parameter: str

None

date: str

None

value: float | None

None

distance: float | None

None

taken_station_id: str | None

None

class wetterdienst.model.result._SummarizedValuesDict

Bases: typing_extensions.TypedDict

Format summarized values as dictionary.

Initialization

Initialize self. See help(type(self)) for accurate signature.

metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]

None

stations: typing_extensions.NotRequired[list[wetterdienst.model.result._Station]]

None

values: list[wetterdienst.model.result._SummarizedValuesItemDict]

None

class wetterdienst.model.result._SummarizedValuesOgcFeature

Bases: typing_extensions.TypedDict

Format summarized values as OGC feature.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[Feature]

None

properties: wetterdienst.model.result._InterpolatedOrSummarizedOgcFeatureProperties

None

geometry: wetterdienst.model.result._OgcFeatureGeometry

None

stations: list[wetterdienst.model.result._Station]

None

values: list[wetterdienst.model.result._SummarizedValuesItemDict]

None

class wetterdienst.model.result._SummarizedValuesOgcFeatureCollectionData

Bases: typing_extensions.TypedDict

Format summarized values as OGC feature collection data.

Initialization

Initialize self. See help(type(self)) for accurate signature.

type: Literal[FeatureCollection]

None

features: list[wetterdienst.model.result._SummarizedValuesOgcFeature]

None

class wetterdienst.model.result._SummarizedValuesOgcFeatureCollection

Bases: typing_extensions.TypedDict

Format summarized values as OGC feature collection.

Initialization

Initialize self. See help(type(self)) for accurate signature.

metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]

None

data: wetterdienst.model.result._SummarizedValuesOgcFeatureCollectionData

None

class wetterdienst.model.result.SummarizedValuesResult

Bases: wetterdienst.model.result._ValuesResult

Calculate summary of stations and parameters.

stations: wetterdienst.model.result.StationsResult

None

df: polars.DataFrame

None

latlon: tuple[float, float]

None

to_ogc_feature_collection(*, with_metadata: bool = False, **_kwargs) wetterdienst.model.result._SummarizedValuesOgcFeatureCollection

Export summarized values as OGC feature collection.

to_plot(**_kwargs: dict) plotly.graph_objects.Figure

Create a plotly figure from the values DataFrame.

_to_image(fmt: Literal[html, png, jpg, webp, svg, pdf], width: int | None = None, height: int | None = None, scale: float | None = None, **kwargs: dict) bytes | str

Create an image from the plotly figure.

This method is used by .to_image() to create an image for summarized values from the plotly figure.

wetterdienst.model.metadata#

Metadata models for a provider.

Module Contents#

Classes#

ParameterModel

Parameter model for a provider.

DatasetModel

Dataset model for a provider.

ResolutionModel

Resolution model for a provider.

MetadataModel

Metadata model for a provider.

ParameterSearch

Dataclass to hold a search for a parameter.

Functions#

build_metadata_model

Build a MetadataModel from a dictionary.

parse_parameters

Parse parameters, either from string or tuple or MetadataModel or sequence of those.

Data#

log

POSSIBLE_SEPARATORS

DATASET_NAME_DEFAULT

API#

wetterdienst.model.metadata.log

‘getLogger(…)’

wetterdienst.model.metadata.POSSIBLE_SEPARATORS

(‘/’, ‘.’, ‘:’)

wetterdienst.model.metadata.DATASET_NAME_DEFAULT

‘data’

class wetterdienst.model.metadata.ParameterModel(/, **data: Any)

Bases: pydantic.BaseModel

Parameter model for a provider.

A provider declares only what it itself knows: the canonical name as a foreign key into PARAMETERS, the source’s own name_original and the source’s unit. The unit_type is a property of the measured quantity rather than of the provider, so it is read from the canonical table instead of being declared – see unit_type below.

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

model_config

‘ConfigDict(…)’

name: str

None

name_original: str

None

unit: str

None

description: str | None

None

dataset: pydantic.SkipValidation[DatasetModel]

‘Field(…)’

property unit_type: wetterdienst.metadata.unit_type.UnitType

The unit type of the measured quantity, from the canonical parameter table.

Resolved on access rather than at import, so an unknown name is caught by tests/test_api.py::test_metadata_parameter_table rather than by every user paying a table lookup per declaration on every interpreter start.

A plain property rather than a computed_field, so it does not reappear in model_dump(). It is derived from name, and re-emitting it per declaration would put back at the serialization layer the duplication this model exists to remove. Callers that want it from a dump can look the name up in PARAMETERS; discover() and the REST and CLI responses build their own dicts and report it as before.

__eq__(other: object) bool

Compare two parameters.

class wetterdienst.model.metadata.DatasetModel(**data: dict)

Bases: pydantic.BaseModel

Dataset model for a provider.

Initialization

Initialize the dataset model.

name: str

None

name_original: str

None

grouped: bool

None

periods: list[wetterdienst.metadata.period.Period]

None

description: str | None

None

date_required: bool

None

parameters: list[wetterdienst.model.metadata.ParameterModel]

None

resolution: pydantic.SkipValidation[ResolutionModel]

‘Field(…)’

__eq__(other: object) bool

Compare two datasets.

__getitem__(item: str | int) wetterdienst.model.metadata.ParameterModel

Get a parameter by name.

__getattr__(item: str) wetterdienst.model.metadata.ParameterModel

Get a parameter by name.

__iter__() collections.abc.Iterator[wetterdienst.model.metadata.ParameterModel]

Iterate over all parameters.

class wetterdienst.model.metadata.ResolutionModel(**data: dict)

Bases: pydantic.BaseModel

Resolution model for a provider.

Initialization

Initialize the resolution model.

name: str

None

name_original: str

None

value: wetterdienst.metadata.resolution.Resolution

‘Field(…)’

periods: list[wetterdienst.metadata.period.Period] | None

None

description: str | None

None

date_required: bool | None

None

datasets: list[wetterdienst.model.metadata.DatasetModel]

None

classmethod validate_datasets(v: list[dict], validation_info: pydantic_core.core_schema.ValidationInfo) list[wetterdienst.model.metadata.DatasetModel]

Validate datasets and set resolution for each dataset.

__getitem__(item: str | int) wetterdienst.model.metadata.DatasetModel

Get a dataset by name.

__getattr__(item: str) wetterdienst.model.metadata.DatasetModel

Get a dataset by name.

__iter__() collections.abc.Iterator[wetterdienst.model.metadata.DatasetModel]

Iterate over all datasets.

class wetterdienst.model.metadata.MetadataModel(/, **data: Any)

Bases: pydantic.BaseModel

Metadata model for a provider.

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

name_short: str

None

name_english: str

None

name_local: str

None

country: str

None

copyright: str

None

url: str

None

kind: Literal[observation, forecast, derived]

None

timezone: pydantic_extra_types.timezone_name.TimeZoneName

None

timezone_data: pydantic_extra_types.timezone_name.TimeZoneName | Literal[dynamic]

None

auth: bool

False

resolutions: list[wetterdienst.model.metadata.ResolutionModel]

None

__getitem__(item: str | int) wetterdienst.model.metadata.ResolutionModel

Get a resolution by name.

__getattr__(item: str) wetterdienst.model.metadata.ResolutionModel

Get a resolution by name.

Alternatively, this still finds any other attribute that is not a resolution.

__iter__() collections.abc.Iterator[wetterdienst.model.metadata.ResolutionModel]

Iterate over all resolutions.

search_parameter(parameter_search: wetterdienst.model.metadata.ParameterSearch) list[wetterdienst.model.metadata.ParameterModel]

Search for a parameter in the metadata.

wetterdienst.model.metadata.build_metadata_model(metadata: dict, name: str) wetterdienst.model.metadata.MetadataModel

Build a MetadataModel from a dictionary.

Attaches the descriptions kept in metadata.source_descriptions, for parameters, datasets and resolutions alike. Those are the curated descriptions the provider docs tables have always carried. A description a provider module already declares wins, since that is a transcription of the source’s own wording and is only kept where it says at least as much as the curated text. DERIVED_DESCRIPTIONS fills only what no source supplies at all.

class wetterdienst.model.metadata.ParameterSearch

Dataclass to hold a search for a parameter.

resolution: str

None

dataset: str

None

parameter: str | None

None

classmethod parse(value: str | collections.abc.Iterable[str] | wetterdienst.model.metadata.DatasetModel | wetterdienst.model.metadata.ParameterModel) wetterdienst.model.metadata.ParameterSearch

Parse a string or tuple or DatasetModel or ParameterModel into a ParameterSearch object.

concat() str

Concatenate resolution, dataset and parameter with ‘/’.

wetterdienst.model.metadata.parse_parameters(parameters: wetterdienst.model.request._PARAMETER_TYPE, metadata: wetterdienst.model.metadata.MetadataModel) list[wetterdienst.model.metadata.ParameterModel]

Parse parameters, either from string or tuple or MetadataModel or sequence of those.

wetterdienst.io.export#

Export data to various formats.

Module Contents#

Classes#

ExportMixin

Postprocessing data.

Functions#

convert_datetimes

Convert all datetime columns to ISO format.

Data#

log

API#

wetterdienst.io.export.log

‘getLogger(…)’

class wetterdienst.io.export.ExportMixin

Postprocessing data.

This aids in collecting, filtering, formatting and emitting data acquired through the core machinery.

df: polars.DataFrame

None

filter_by_sql(sql: str) polars.DataFrame

Filter df using an SQL query WHERE clause.

abstractmethod to_dict(*args: Any, **kwargs: Any) dict

Convert station information into dictionary format.

abstractmethod to_json(*args: Any, **kwargs: Any) str

Convert station information into JSON format.

abstractmethod to_ogc_feature_collection(*args: Any, with_metadata: bool, **kwargs: Any) dict

Convert station information into OGC Feature Collection format.

Abstract method implementation.

to_geojson(*, with_metadata: bool = False, indent: int | bool | None = 4, **_kwargs: Any) str

Convert station information into GeoJSON format.

Args: with_metadata: Include metadata in GeoJSON indent: Indentation level for JSON output

Returns: GeoJSON string

to_csv(**kwargs: Any) str

Convert DataFrame to CSV format.

Args: **kwargs: Additional arguments passed to the CSV writer

Returns: CSV string

abstractmethod to_plot(**kwargs: Any) plotly.graph_objs.Figure

Create a plotly figure from the DataFrame.

abstractmethod _to_image(**kwargs: Any) bytes | str

Create an image from the plotly figure.

to_image(**kwargs: Any) bytes | str

Create an image from the plotly figure.

Args: **kwargs: Additional arguments passed to the image creation method

Returns: Image data as bytes or string

to_format(fmt: str, **kwargs: Any) str | bytes

Format data according to the specified format.

The formatting is done by one of the following methods:

  • to_json

  • to_csv

  • to_geojson

  • to_image

Args: fmt: Output format **kwargs: Additional arguments passed to the formatting method

Returns: Formatted data

static _filter_by_sql(df: polars.DataFrame, sql: str) polars.DataFrame

Filter df using an SQL query WHERE clause.

This implementation is based on DuckDB, so please have a look at its SQL documentation.

  • https://duckdb.org/docs/sql/introduction

Args: df: DataFrame to filter sql: SQL WHERE clause

Returns: Filtered DataFrame

to_target(target: str, if_exists: Literal[replace, append, fail, skip] = 'replace') None

Emit data to a target.

The target is identified by a connection string.

Examples:

  • duckdb://dwd.duckdb?table=weather

  • influxdb://localhost/?database=dwd&table=weather

  • crate://localhost/?database=dwd&table=weather

Dispatch data to different data sinks. Currently, SQLite, DuckDB, InfluxDB and CrateDB are implemented. However, through the SQLAlchemy layer, it should actually work with any supported SQL database.

  • https://docs.sqlalchemy.org/en/13/dialects/

Args: target: Connection string if_exists: Behavior when the target already exists. Options: ‘replace’, ‘append’, ‘fail’, ‘skip’. Default is ‘replace’. - ‘replace’: Drop and recreate the target (default, backward compatible) - ‘append’: Append data to the target - ‘fail’: Raise error if target exists - ‘skip’: Do not write if target exists (only for supported backends)

Raises: KeyError: Unknown export

Returns: None (data is emitted to the target)

wetterdienst.io.export.convert_datetimes(df: polars.DataFrame) polars.DataFrame

Convert all datetime columns to ISO format.

wetterdienst.settings#

Settings for the wetterdienst package.

Module Contents#

Classes#

Auth

Authentication credentials for providers requiring API keys.

Settings

Settings for the wetterdienst package.

Functions#

_default_geo_station_distance

Build the per-parameter search radius from the canonical parameter table.

Data#

log

_UNIT_CONVERTER_TARGETS

_STATION_DISTANCE_HOMOGENEOUS

_STATION_DISTANCE_HETEROGENEOUS

API#

wetterdienst.settings.log

‘getLogger(…)’

wetterdienst.settings._UNIT_CONVERTER_TARGETS

‘keys(…)’

class wetterdienst.settings.Auth(/, **data: Any)

Bases: pydantic.BaseModel

Authentication credentials for providers requiring API keys.

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

aemet: str | None

‘Field(…)’

knmi: str | None

‘Field(…)’

metno_frost: tuple[str, str] | None

‘Field(…)’

ceda: tuple[str, str] | None

‘Field(…)’

classmethod validate_metno_frost(value: tuple[str, str] | str | None) tuple[str, str] | None
classmethod validate_ceda(value: tuple[str, str] | str | None) tuple[str, str] | None

Parse the CEDA (username, password) pair, e.g. from WD_AUTH__CEDA=username:password.

wetterdienst.settings._STATION_DISTANCE_HOMOGENEOUS

40.0

wetterdienst.settings._STATION_DISTANCE_HETEROGENEOUS

20.0

wetterdienst.settings._default_geo_station_distance() collections.defaultdict[str, float]

Build the per-parameter search radius from the canonical parameter table.

Which names get the shorter radius used to be written out here, a copy of a classification the table already holds. Only those names are put in the dict; the default factory answers for every other parameter, so the setting a user sees and overrides stays the short list of exceptions rather than all 514 names.

class wetterdienst.settings.Settings(_case_sensitive: bool | None = None, _nested_model_default_partial_update: bool | None = None, _env_prefix: str | None = None, _env_prefix_target: pydantic_settings.sources.EnvPrefixTarget | None = None, _env_file: pydantic_settings.sources.DotenvType | None = ENV_FILE_SENTINEL, _env_file_encoding: str | None = None, _env_ignore_empty: bool | None = None, _env_nested_delimiter: str | None = None, _env_nested_max_split: int | None = None, _env_parse_none_str: str | None = None, _env_parse_enums: bool | None = None, _cli_prog_name: str | None = None, _cli_parse_args: bool | list[str] | tuple[str, ...] | None = None, _cli_settings_source: pydantic_settings.sources.CliSettingsSource[Any] | None = None, _cli_parse_none_str: str | None = None, _cli_hide_none_type: bool | None = None, _cli_avoid_json: bool | None = None, _cli_enforce_required: bool | None = None, _cli_use_class_docs_for_groups: bool | None = None, _cli_show_env_vars: bool | None = None, _cli_exit_on_error: bool | None = None, _cli_prefix: str | None = None, _cli_flag_prefix_char: str | None = None, _cli_implicit_flags: bool | Literal[dual, toggle] | None = None, _cli_ignore_unknown_args: bool | None = None, _cli_kebab_case: bool | Literal[all, no_enums] | None = None, _cli_shortcuts: collections.abc.Mapping[str, str | list[str]] | None = None, _secrets_dir: pydantic_settings.sources.PathType | None = None, _build_sources: tuple[tuple[pydantic_settings.sources.PydanticBaseSettingsSource, ...], dict[str, Any]] | None = None, **values: Any)

Bases: pydantic_settings.BaseSettings

Settings for the wetterdienst package.

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

model_config

‘SettingsConfigDict(…)’

cache_disable: bool

‘Field(…)’

cache_dir: pathlib.Path

‘Field(…)’

fsspec_client_kwargs: dict

‘Field(…)’

auth: wetterdienst.settings.Auth

‘Field(…)’

use_certifi: bool

‘Field(…)’

read_bufr: bool

‘Field(…)’

ts_humanize: bool

True

ts_shape: Literal[wide, long]

‘long’

ts_convert_units: bool

True

ts_unit_targets: dict[str, str]

‘Field(…)’

ts_skip_empty: bool

False

ts_skip_threshold: float

0.95

ts_skip_criteria: Literal[min, mean, max]

‘min’

ts_complete: bool

False

ts_drop_nulls: bool

True

ts_geo_station_distance: collections.defaultdict[str, float]

‘Field(…)’

ts_geo_use_nearby_station_distance: Annotated[float, Field(strict=True, ge=0)] | None

1.0

ts_geo_min_gain_of_value_pairs: Annotated[float, Field(strict=True, ge=0)]

0.1

ts_geo_num_additional_stations: Annotated[int, Field(strict=True, ge=0)]

3

classmethod validate_ts_unit_targets_before(values: dict[str, str] | None) dict[str, str]

Validate the unit targets.

classmethod validate_ts_unit_targets_after(values: dict[str, str]) dict[str, str]

Validate the unit targets.

classmethod validate_ts_geo_station_distance(values: dict[str, float] | None) dict[str, float]

Validate the interpolation station distance settings.

property ts_tidy: bool

Return whether the time series is in tidy format.

validate() wetterdienst.settings.Settings

Validate the settings.

__repr__() str

Return the settings as a JSON string.

__str__() str

Return the settings as a string.

wetterdienst.util.geo#

Geo utilities for the wetterdienst package.

Module Contents#

Functions#

derive_nearest_neighbours

Obtain the nearest neighbours using a simple distance computation.

convert_dm_to_dd

Convert degree minutes (floats) to decimal degree.

convert_dms_string_to_dd

Convert degree minutes seconds (string) to decimal degree.

Data#

pc

EARTH_RADIUS_IN_KM

API#

wetterdienst.util.geo.pc: Any

None

wetterdienst.util.geo.EARTH_RADIUS_IN_KM

6371

wetterdienst.util.geo.derive_nearest_neighbours(latitudes: pyarrow.Array, longitudes: pyarrow.Array, q_lat: float, q_lon: float) list[float]

Obtain the nearest neighbours using a simple distance computation.

Args: latitudes: latitudes in degree longitudes: longitudes in degree q_lat: latitude of the query point q_lon: longitude of the query point

Returns: Tuple of distances and ranks of nearest to most distant station

wetterdienst.util.geo.convert_dm_to_dd(dm: polars.Series) polars.Series

Convert degree minutes (floats) to decimal degree.

Args: dm: Series with degree minutes as float

Returns: Series with decimal degree

wetterdienst.util.geo.convert_dms_string_to_dd(dms: polars.Series) polars.Series

Convert degree minutes seconds (string) to decimal degree.

Args: dms: Series with degree minutes seconds as string

Returns: Series with decimal degree

wetterdienst.util.network#

Network utilities for the wetterdienst package.

Module Contents#

Classes#

File

File object for the network utilities.

FileDirCache

File-based cache for FSSPEC.

HTTPFileSystem

HTTPFileSystem with cache support.

NetworkFilesystemManager

Manage multiple FSSPEC instances keyed by cache expiration time.

Functions#

_create_ssl_context

Create an SSL context optionally using certifi certificates.

list_remote_files_fsspec

Create a listing of all files of a given path on the server.

list_remote_directory_fsspec

List the immediate contents (files and subdirectories) of a given path on the server, non-recursively.

download_file

Download a specified file from the server.

download_files

Download multiple files from the server concurrently.

Data#

log

API#

wetterdienst.util.network.log

‘getLogger(…)’

wetterdienst.util.network._create_ssl_context(*, use_certifi: bool) ssl.SSLContext | None

Create an SSL context optionally using certifi certificates.

Args: use_certifi: If True, use certifi certificate bundle instead of system certificates.

Returns: An SSL context configured with certifi certificates if requested, None otherwise.

class wetterdienst.util.network.File

File object for the network utilities.

url: str

None

The URL of the file.

property filename: str

The filename of the file.

content: io.BytesIO | Exception

None

The content of the file as a BytesIO object.

status: int

None

The status code of the file download, if available.

raise_if_exception() None

Raise an exception if the content is not a BytesIO object.

For NoInternetError, logs at debug level and returns silently instead of raising, allowing callers to return empty frames rather than propagating the error.

property is_no_internet_error: bool

Check if the content is a NoInternetError.

property nbytes: int

Return the number of bytes in the file content.

property is_empty: bool

Check if the file content is empty.

class wetterdienst.util.network.FileDirCache(listings_expiry_time: float, *, use_listings_cache: bool, listings_cache_location: pathlib.Path | None = None)

Bases: collections.abc.MutableMapping

File-based cache for FSSPEC.

Initialization

Initialize the FileDirCache.

Args: listings_expiry_time: Time in seconds that a listing is considered valid. use_listings_cache: If False, this cache never returns items, but always reports KeyError. listings_cache_location: Directory path at which the listings cache file is stored.

__getitem__(item: str) io.BytesIO

Draw item as fileobject from cache, retry if timeout occurs.

clear() None

Clear cache.

__len__() int

Return number of items in cache.

__contains__(item: object) bool

Check if item is in cache and not expired.

__setitem__(key: str, value: io.BytesIO) None

Store fileobject in cache.

__delitem__(key: str) None

Remove item from cache.

__iter__() collections.abc.Iterator[str]

Iterate over keys in cache.

__reduce__() tuple

Return state information for pickling.

class wetterdienst.util.network.HTTPFileSystem(/, *, use_listings_cache: bool, listings_expiry_time: float, listings_cache_location: pathlib.Path | None = None, use_certifi: bool = False, **kwargs)

Bases: fsspec.implementations.http.HTTPFileSystem

HTTPFileSystem with cache support.

Initialization

Initialize the HTTPFileSystem.

Args: use_listings_cache: If False, this cache never returns items, but always reports KeyError, listings_expiry_time: Time in seconds that a listing is considered valid. If None, listings_cache_location: Directory path at which the listings cache file is stored. If None, use_certifi: If True, use certifi certificate bundle instead of system certificates. *args: Additional arguments. **kwargs: Additional keyword arguments.

class wetterdienst.util.network.NetworkFilesystemManager

Manage multiple FSSPEC instances keyed by cache expiration time.

Each thread gets its own set of filesystem instances to avoid thread-safety issues with WholeFileCacheFileSystem’s in-memory metadata cache.

_thread_local: ClassVar[threading.local]

‘local(…)’

classmethod _get_filesystems() dict[str, wetterdienst.util.network.HTTPFileSystem | fsspec.implementations.cached.WholeFileCacheFileSystem]

Return the per-thread filesystem registry.

static _client_kwargs_suffix(client_kwargs: dict | None) str

Return a short stable hash suffix that distinguishes different client_kwargs (e.g. auth headers).

static resolve_ttl(cache_expiry: wetterdienst.metadata.cache.CacheExpiry) tuple[str, float | int | Literal[False]]

Resolve the cache expiration time.

Args: cache_expiry: The cache expiration time.

Returns: The cache expiration time as name and value.

classmethod register(cache_dir: pathlib.Path, cache_expiry: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.NO_CACHE, client_kwargs: dict | None = None, *, cache_disable: bool, use_certifi: bool = False) None

Register a new filesystem instance for a given cache expiration time.

Args: cache_dir: The cache directory to use for the filesystem. cache_expiry: The cache expiration time. client_kwargs: Additional keyword arguments for the client. cache_disable: If True, the cache is disabled. use_certifi: If True, use certifi certificate bundle instead of system certificates.

Returns: None

classmethod get(cache_dir: pathlib.Path, cache_expiry: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.NO_CACHE, client_kwargs: dict | None = None, *, cache_disable: bool, use_certifi: bool = False) wetterdienst.util.network.HTTPFileSystem | fsspec.implementations.cached.WholeFileCacheFileSystem

Get a filesystem instance for a given cache expiration time.

Args: cache_dir: The cache directory to use for the filesystem. cache_expiry: The cache expiration time. client_kwargs: Additional keyword arguments for the client. cache_disable: If True, the cache is disabled use_certifi: If True, use certifi certificate bundle instead of system certificates.

Returns: The filesystem instance.

wetterdienst.util.network.list_remote_files_fsspec(url: str, settings: wetterdienst.settings.Settings, cache_expiry: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.FILEINDEX) list[str]

Create a listing of all files of a given path on the server.

The default ttl with CacheExpiry.FILEINDEX is “5 minutes”.

Args: url: The URL to list files from. settings: The settings to use for the listing. cache_expiry: The cache expiration time.

Returns: A list of all files on the server

wetterdienst.util.network.list_remote_directory_fsspec(url: str, settings: wetterdienst.settings.Settings, cache_expiry: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.FILEINDEX) list[dict]

List the immediate contents (files and subdirectories) of a given path on the server, non-recursively.

Unlike list_remote_files_fsspec, this does not descend into subdirectories, which is useful for servers exposing a deeply nested directory tree where the folder names themselves carry enough information (e.g. a date range) to decide which subdirectories are actually worth descending into.

Args: url: The URL to list the contents of. settings: The settings to use for the listing. cache_expiry: The cache expiration time.

Returns: A list of fsspec detail dicts (with “name” and “type” keys, among others) for each entry.

wetterdienst.util.network.download_file(url: str, cache_dir: pathlib.Path, ttl: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.NO_CACHE, client_kwargs: dict | None = None, *, cache_disable: bool = False, use_certifi: bool = False) wetterdienst.util.network.File

Download a specified file from the server.

Args: url: The URL of the file to download. cache_dir: The cache directory to use for the filesystem. ttl: The cache expiration time. client_kwargs: Additional keyword arguments for the client. cache_disable: If True, the cache is disabled. use_certifi: If True, use certifi certificate bundle instead of system certificates.

Returns: A BytesIO object containing the downloaded file.

wetterdienst.util.network.download_files(urls: list[str], cache_dir: pathlib.Path, ttl: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.NO_CACHE, client_kwargs: dict | None = None, *, cache_disable: bool = False, use_certifi: bool = False) list[wetterdienst.util.network.File]

Download multiple files from the server concurrently.