Core#
wetterdienst.core.interpolate#
Interpolation for weather data.
Module Contents#
Functions#
|
Get the interpolated DataFrame for the given request and location. |
|
Request the stations for the interpolation. |
|
Apply the station values to the parameter data. |
|
Calculate the interpolation for the given data. |
|
Get all valid station groups that cover the given point. |
|
Get the station group ids that are a subset of the given values. |
|
Apply interpolation to a row of data. |
Data#
|
API#
- wetterdienst.core.interpolate.log
‘getLogger(…)’
- wetterdienst.core.interpolate.get_interpolated_df(request: wetterdienst.model.request.TimeseriesRequest, latitude: float, longitude: float) polars.DataFrame
Get the interpolated DataFrame for the given request and location.
- wetterdienst.core.interpolate.request_stations(request: wetterdienst.model.request.TimeseriesRequest, latitude: float, longitude: float, utm_x: float, utm_y: float) tuple[dict, dict]
Request the stations for the interpolation.
Args: request: TimeseriesRequest object latitude: latitude of the point to interpolate longitude: longitude of the point to interpolate utm_x: longitude in UTM of the point to interpolate utm_y: latitude in UTM of the point to interpolate
Returns: tuple containing the stations dict and the parameter dict
- wetterdienst.core.interpolate.apply_station_values_per_parameter(result_df: polars.DataFrame, stations_ranked: wetterdienst.model.result.StationsResult, param_dict: dict, station: dict, *, valid_station_groups_exists: bool) None
Apply the station values to the parameter data.
Args: result_df: DataFrame containing the station values stations_ranked: stations_result with stations ranked by distance param_dict: dict containing the parameter data station: dict containing the station data min_gain_of_value_pairs: minimum gain of value pairs to add a station num_additional_stations: number of additional stations to add if the gain is not reached valid_station_groups_exists: bool indicating if valid station groups exist
Returns: None - the parameter data is updated in place
- wetterdienst.core.interpolate.calculate_interpolation(utm_x: float, utm_y: float, stations_dict: dict, param_dict: dict, use_nearby_station_distance: float | None) polars.DataFrame
Calculate the interpolation for the given data.
Args: utm_x: longitude in UTM utm_y: latitude in UTM stations_dict: dict containing the station data including the location param_dict: dict containing the parameter data use_nearby_station_distance: distance in km to use nearby stations for interpolation
Returns: DataFrame containing the interpolated data
- wetterdienst.core.interpolate.get_valid_station_groups(stations_dict: dict, utm_x: float, utm_y: float) queue.Queue
Get all valid station groups that cover the given point.
Args: stations_dict: dict containing the station data including the location utm_x: longitude in UTM utm_y: latitude in UTM
Returns: Queue containing the valid station groups
- wetterdienst.core.interpolate.get_station_group_ids(valid_station_groups: queue.Queue, vals_index: frozenset) list
Get the station group ids that are a subset of the given values.
- wetterdienst.core.interpolate.apply_interpolation(row: dict, stations_dict: dict, valid_station_groups: queue.Queue, resolution: str, dataset: str, parameter: str, utm_x: float, utm_y: float, nearby_stations: list[str]) tuple[str, str, str, float | None, float | None, list[str]]
Apply interpolation to a row of data.
Args: row: dict containing the data across collected stations for a specific date stations_dict: dict containing the station data including the location valid_station_groups: Queue containing the valid station groups to use for interpolation resolution: resolution name dataset: dataset name parameter: parameter name utm_x: longitude in UTM utm_y: latitude in UTM nearby_stations: list of nearby stations
Returns: tuple containing the resolution name, dataset name, parameter name, interpolated value, mean distance of the stations used for interpolation and the station ids used for interpolation
wetterdienst.model.request#
Core for timeseries information of a source.
Module Contents#
Classes#
|
Core class for timeseries information of a source. |
Data#
|
|
|
|
|
|
|
|
|
API#
- wetterdienst.model.request.log
‘getLogger(…)’
- wetterdienst.model.request.EARTH_RADIUS_KM
6371
- wetterdienst.model.request._PARAMETER_TYPE_SINGULAR
None
- wetterdienst.model.request._PARAMETER_TYPE
None
- wetterdienst.model.request._DATETIME_TYPE
None
- class wetterdienst.model.request.TimeseriesRequest
Core class for timeseries information of a source.
- metadata: wetterdienst.model.metadata.MetadataModel
‘field(…)’
- _values: wetterdienst.model.values.TimeseriesValues
‘field(…)’
- _history: wetterdienst.model.history.TimeseriesHistory
‘field(…)’
- parameters: wetterdienst.model.request._PARAMETER_TYPE
None
- start_date: wetterdienst.model.request._DATETIME_TYPE
None
- end_date: wetterdienst.model.request._DATETIME_TYPE
None
- settings: wetterdienst.settings.Settings | dict
‘field(…)’
- __post_init__() None
Post init method to validate the settings and convert the timestamps.
- _base_columns: ClassVar
(‘resolution’, ‘dataset’, ‘station_id’, ‘start_date’, ‘end_date’, ‘latitude’, ‘longitude’, ‘height’,…
- interpolatable_parameters: ClassVar
None
- static _parse_station_id(series: polars.Series) polars.Series
Parse station_id column to string.
Args: series: Series containing station ids.
Returns: pl.Series: Series with station ids as strings.
- static convert_timestamps(start_date: wetterdienst.model.request._DATETIME_TYPE, end_date: wetterdienst.model.request._DATETIME_TYPE) tuple[None, None] | tuple[datetime.datetime, datetime.datetime]
Convert timestamps to datetime objects.
Args: start_date: Start date of the request. end_date: End date of the request.
Returns: tuple[None, None] | tuple[dt.datetime, dt.datetime]: Start and end date of the request.
- classmethod is_configured() bool
Return True if this provider’s required credentials are present (env var / settings).
This is a cheap, offline check. Override in subclasses that require authentication.
- classmethod is_valid(settings: wetterdienst.settings.Settings | None = None) bool
Return True if the provider’s credentials are valid (authenticated successfully).
This may perform a lightweight network probe and should cache the result. Only called when is_configured() is True. Override in auth-requiring subclasses. The optional
settingsargument lets callers pass a custom Settings instance; implementations that ignore it fall back to Settings() internally.
- classmethod discover(resolutions: str | wetterdienst.metadata.resolution.Resolution | wetterdienst.model.metadata.ResolutionModel | collections.abc.Sequence[str | wetterdienst.metadata.resolution.Resolution | wetterdienst.model.metadata.ResolutionModel] | None = None, datasets: str | wetterdienst.model.metadata.DatasetModel | collections.abc.Sequence[str | wetterdienst.model.metadata.DatasetModel] | None = None) dict
Discover metadata for the given resolutions and datasets.
Each level carries its own description, so the shape has a place for one::
{resolution: {"description": ..., "datasets": {dataset: {"description": ..., "parameters": [{"name": ..., "name_original": ..., "unit_type": ..., "unit": ..., "description": ...}]}}}}Args: resolutions: Resolutions to discover metadata for. datasets: Datasets to discover metadata for.
Returns: dict: Metadata for the given resolutions and datasets.
- static _coerce_meta_fields(df: polars.DataFrame) polars.DataFrame
Coerce metadata fields to the correct types.
- abstractmethod _all() polars.LazyFrame
Implement this method to get all stations.
Returns: pl.LazyFrame: All stations.
- all() wetterdienst.model.result.StationsResult
Get all stations.
Returns: StationsResult: All stations.
- filter_by_station_id(station_id: str | tuple[str, ...] | list[str]) wetterdienst.model.result.StationsResult
Filter stations by station_id.
Args: station_id: Station id or list of station ids.
Returns: StationsResult: Filtered stations.
- filter_by_name(name: str, rank: int = 1, threshold: float = 0.8) wetterdienst.model.result.StationsResult
Filter stations by name.
Args: name: Name of the station. rank: Maximum number of matches to return, best score first (default 1). threshold: Threshold for the fuzzy search.
Returns: StationsResult: Filtered stations.
- filter_by_rank(latlon: tuple[float, float], rank: int) wetterdienst.model.result.StationsResult
Filter stations by rank.
Rank is defined by distance to the requested point. The resulting
StationsResult.dfholds all stations sorted by distance, not justrankrows: because we cannot know upfront which stations actually carry data for the request, theranklimit is applied lazily while collecting values. Value collection walks the distance-sorted stations and stops oncerankstations with data (per thets_skip_empty/ts_skip_threshold/ts_skip_criteriasettings) have been consumed. The stations that ended up contributing values are then exposed viaValuesResult.df_stations.In other words, use
stations.values.all().df_stations(notstations.df) to see therankclosest stations that actually returned data. Setts_skip_empty=Falseto simply take therankclosest stations regardless of data availability.Args: latlon: Latitude and longitude for the requested point. rank: Number of stations requested.
Returns: StationsResult: Stations sorted by distance (see note above on
rank).
- filter_by_distance(latlon: tuple[float, float], distance: float, unit: str = 'km') wetterdienst.model.result.StationsResult
Filter stations by distance.
Args: latlon: Latitude and longitude for the requested point. distance: Maximum distance to the requested point. unit: Unit of the distance.
Returns: StationsResult: Filtered stations.
- filter_by_bbox(left: float, bottom: float, right: float, top: float) wetterdienst.model.result.StationsResult
Filter stations by bounding box.
Args: left: Left border of the bounding box. bottom: Bottom border of the bounding box. right: Right border of the bounding box. top: Top border of the bounding box.
Returns: StationsResult: Filtered stations.
- filter_by_sql(sql: str) wetterdienst.model.result.StationsResult
Filter stations by SQL query.
Args: sql: SQL query to filter stations by.
Returns: StationsResult: Filtered stations.
- interpolate(latlon: tuple[float, float]) wetterdienst.model.result.InterpolatedValuesResult
Interpolate values across multiple stations.
Interpolation means we interpolate the values of the closest available stations to the requested point.
Args: latlon: Latitude and longitude for the requested point.
Returns: InterpolatedValuesResult: Interpolated values.
- interpolate_by_station_id(station_id: str) wetterdienst.model.result.InterpolatedValuesResult
Use .interpolate with station_id instead of latlon.
- summarize(latlon: tuple[float, float]) wetterdienst.model.result.SummarizedValuesResult
Summarize values across multiple stations.
Summarize means we take any available data of the closest station as representative for the timestamp.
- summarize_by_station_id(station_id: str) wetterdienst.model.result.SummarizedValuesResult
Use .summarize with station_id instead of latlon.
- _get_latlon_by_station_id(station_id: str) tuple[float, float]
Get latlon for a station_id.
Used for .summary/.interpolate. Typically, we expect a latlon tuple of floats, but we want users to be able to request for a station id as well.
- static _create_station_id_from_string(string: str) str
Create station id from string.
Used for interpolation and summarization data
wetterdienst.model.values#
Core for sources of timeseries where data is related to a station.
Module Contents#
Classes#
|
Core for sources of timeseries where data is related to a station. |
Data#
|
API#
- wetterdienst.model.values.log
‘getLogger(…)’
- class wetterdienst.model.values.TimeseriesValues
Bases:
abc.ABCCore for sources of timeseries where data is related to a station.
- sr: wetterdienst.model.result.StationsResult
None
- stations_counter: int
0
- stations_collected: list[str]
‘field(…)’
- unit_converter: wetterdienst.model.unit.UnitConverter
‘field(…)’
- _date_fields: ClassVar
[‘date’, ‘start_date’, ‘end_date’]
- __post_init__() None
Post-initialization of the TimeseriesValues object.
- classmethod from_stations(stations: wetterdienst.model.result.StationsResult) wetterdienst.model.values.TimeseriesValues
Create a new instance of the class from a StationsResult object.
- property _meta_fields: dict[str, Any]
Get metadata fields for the DataFrame.
- property timezone_data: str
Get timezone data for the station.
- _adjust_start_end_date(start_date: datetime.datetime, end_date: datetime.datetime, tzinfo: zoneinfo.ZoneInfo, resolution: wetterdienst.metadata.resolution.Resolution) tuple[datetime.datetime, datetime.datetime]
Adjust start and end date for a given resolution.
- _get_complete_dates(start_date: datetime.datetime, end_date: datetime.datetime, resolution: wetterdienst.metadata.resolution.Resolution) polars.Series
Get a complete date range for a given start and end date and resolution.
- _get_timezone_from_station(station_id: str) str
Get timezone information for explicit station.
This is used to set the correct timezone for the timestamps of the returned values.
- _get_base_df(start_date: datetime.datetime, end_date: datetime.datetime, resolution: wetterdienst.metadata.resolution.Resolution) polars.DataFrame
Create a base DataFrame with all dates for a given station.
- _convert_units(df: polars.DataFrame, dataset: wetterdienst.model.metadata.DatasetModel) polars.DataFrame
Convert values to metric units with help of conversion factors.
- _create_conversion_lambdas(dataset: wetterdienst.model.metadata.DatasetModel) dict[str, collections.abc.Callable[[Any], Any]]
Create conversion factors based on a given dataset.
- _build_complete_df(df: polars.DataFrame, station_id: str, resolution: wetterdienst.metadata.resolution.Resolution) polars.DataFrame
Build a complete DataFrame with all dates for a given station.
- _organize_df_columns(df: polars.DataFrame, station_id: str, dataset: wetterdienst.model.metadata.DatasetModel) polars.DataFrame
Reorder columns in DataFrame to match the expected order of columns.
- _meta_enum_columns: ClassVar
(‘station_id’, ‘resolution’, ‘dataset’, ‘parameter’)
- classmethod _cast_metadata_to_enum(df: polars.DataFrame) polars.DataFrame
Cast the low-cardinality metadata columns to
Enumto reduce the memory footprint.The categories are taken from the values actually present in the frame (not from the request metadata), so the cast never fails on provider-specific casing or humanization quirks (e.g. WSV emitting
wwhile the metadata declaresW). These columns repeat every row, so integer-backedEnumcodes roughly halve the size of tidy value frames.
- query() collections.abc.Iterator[wetterdienst.model.result.ValuesResult]
Query data for all stations and parameters and return a DataFrame for each station.
- _get_available_datasets(df: polars.DataFrame) list[wetterdienst.model.metadata.DatasetModel]
Extract available datasets for the station.
- _collect_station_data(station_id: str, available_datasets: list[wetterdienst.model.metadata.DatasetModel]) polars.DataFrame
Collect and process data for a single station.
- _process_dataset(station_id: str, dataset: wetterdienst.model.metadata.DatasetModel, parameters: collections.abc.Iterator[wetterdienst.model.metadata.ParameterModel]) polars.DataFrame
Process data for a specific dataset.
- abstractmethod _collect_station_parameter_or_dataset(station_id: str, parameter_or_dataset: wetterdienst.model.metadata.ParameterModel | wetterdienst.model.metadata.DatasetModel) polars.DataFrame
Collect data for a station and a single parameter or dataset.
- _widen_df(df: polars.DataFrame) polars.DataFrame
Widen a dataframe with each row having one timestamp, parameter, value and quality.
Example: date parameter value quality 1971-01-01 precipitation_height 0 0 1971-01-01 temperature_air_mean_2m 10 0
becomes
date precipitation_height qn_precipitation_height 1971-01-01 0 0 temperature_air_mean_2m … 10 …
Args: df: DataFrame with columns date, parameter, value and quality.
Returns: DataFrame with columns date, parameter, value and quality as columns.
- all() wetterdienst.model.result.ValuesResult
Collect all data for all stations and parameters and return a single DataFrame.
- to_target(target: str, if_exists: Literal[replace, append, fail, skip] = 'fail') None
Wrap to_target of all queried results.
- static _humanize(df: polars.DataFrame, humanized_parameters_mapping: dict[str, str]) polars.DataFrame
Humanize parameter names in a DataFrame.
- _create_humanized_parameters_mapping() dict[str, str]
Create mapping of original to humanized parameter names.
- _get_actual_percentage(df: polars.DataFrame) float
Get the percentage of actual values in the DataFrame.
This is used to skip stations with too many missing values.
wetterdienst.model.result#
Result classes for timeseries data.
Module Contents#
Classes#
|
Enumeration for stations filter. |
|
Type definition for provider metadata. |
|
Type definition for producer metadata. |
|
Type definition for metadata. |
|
Type definition for station. |
|
Type definition for dictionary of stations. |
|
Type definition for OGC feature properties. |
|
Type definition for OGC feature geometry. |
|
Type definition for OGC feature of stations. |
|
Type definition for OGC feature collection data of stations. |
|
Type definition for OGC feature collection of stations. |
|
Result class for stations. |
|
Type definition for dictionary of values. |
|
Type definition for dictionary of values. |
|
Result class for values. |
|
Type definition for OGC feature of values. |
|
Type definition for OGC feature collection data of values. |
|
Type definition for OGC feature collection of values. |
|
Result class for values. |
|
Result class for history data. |
|
Type definition for OGC feature properties of interpolated or summarized values. |
|
Type definition for dictionary of interpolated values. |
|
Type definition for dictionary of interpolated values. |
|
Type definition for OGC feature of interpolated values. |
|
Type definition for OGC feature collection data of interpolated values. |
|
Type definition for OGC feature collection of interpolated values. |
|
Result class for interpolated values. |
|
Format summarized values as dictionary. |
|
Format summarized values as dictionary. |
|
Format summarized values as OGC feature. |
|
Format summarized values as OGC feature collection data. |
|
Format summarized values as OGC feature collection. |
|
Calculate summary of stations and parameters. |
API#
- class wetterdienst.model.result.StationsFilter
Bases:
enum.EnumEnumeration for stations filter.
This should help determine why only a subset of stations was returned.
- ALL
‘all’
- BY_STATION_ID
‘by_station_id’
- BY_NAME
‘by_name’
- BY_RANK
‘by_rank’
- BY_DISTANCE
‘by_distance’
- BY_BBOX
‘by_bbox’
- BY_SQL
‘by_sql’
- class wetterdienst.model.result._Provider
Bases:
typing_extensions.TypedDictType definition for provider metadata.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- name_local: str
None
- name_english: str
None
- country: str
None
- copyright: str
None
- url: str
None
- class wetterdienst.model.result._Producer
Bases:
typing_extensions.TypedDictType definition for producer metadata.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- name: str
None
- version: str
None
- repository: str
None
- documentation: str
None
- doi: str
None
- class wetterdienst.model.result._Metadata
Bases:
typing_extensions.TypedDictType definition for metadata.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- provider: wetterdienst.model.result._Provider
None
- producer: wetterdienst.model.result._Producer
None
- class wetterdienst.model.result._Station
Bases:
typing_extensions.TypedDictType definition for station.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- resolution: str
None
- dataset: str
None
- station_id: str
None
- start_date: str | None
None
- end_date: str | None
None
- latitude: float
None
- longitude: float
None
- height: float
None
- name: str
None
- state: str | None
None
- class wetterdienst.model.result._StationsDict
Bases:
typing_extensions.TypedDictType definition for dictionary of stations.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]
None
- stations: list[wetterdienst.model.result._Station]
None
- class wetterdienst.model.result._OgcFeatureProperties
Bases:
typing_extensions.TypedDictType definition for OGC feature properties.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- resolution: str
None
- dataset: str
None
- id: str
None
- name: str
None
- state: str | None
None
- start_date: str | None
None
- end_date: str | None
None
- class wetterdienst.model.result._OgcFeatureGeometry
Bases:
typing_extensions.TypedDictType definition for OGC feature geometry.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[Point]
None
- coordinates: list[float]
None
- class wetterdienst.model.result._StationsOgcFeature
Bases:
typing_extensions.TypedDictType definition for OGC feature of stations.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[Feature]
None
- properties: wetterdienst.model.result._OgcFeatureProperties
None
- geometry: wetterdienst.model.result._OgcFeatureGeometry
None
- class wetterdienst.model.result._StationsOgcFeatureCollectionData
Bases:
typing_extensions.TypedDictType definition for OGC feature collection data of stations.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[FeatureCollection]
None
- features: list[wetterdienst.model.result._StationsOgcFeature]
None
- class wetterdienst.model.result._StationsOgcFeatureCollection
Bases:
typing_extensions.TypedDictType definition for OGC feature collection of stations.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]
None
- data: wetterdienst.model.result._StationsOgcFeatureCollectionData
None
- class wetterdienst.model.result.StationsResult
Bases:
wetterdienst.io.export.ExportMixinResult class for stations.
- stations: wetterdienst.model.request.TimeseriesRequest | wetterdienst.provider.dwd.mosmix.DwdMosmixRequest | wetterdienst.provider.dwd.dmo.DwdDmoRequest
None
- df: polars.DataFrame
None
- df_all: polars.DataFrame
None
- stations_filter: wetterdienst.model.result.StationsFilter
None
- rank: int | None
None
- property settings: wetterdienst.Settings
Get settings for the request.
- property parameters: list[wetterdienst.model.metadata.ParameterModel]
Get parameters from the request.
- property values: wetterdienst.model.values.TimeseriesValues
Get values from the request.
- property history: wetterdienst.model.history.TimeseriesHistory
Get history from the request.
- property start_date: datetime.datetime | None
Get start date from the request.
- property end_date: datetime.datetime | None
Get end date from the request.
- property station_id: polars.Series
Get station IDs from the DataFrame.
- get_metadata() wetterdienst.model.result._Metadata
Get metadata for the provider and producer.
- to_dict(*, with_metadata: bool = False) wetterdienst.model.result._StationsDict
Format station information as dictionary.
Args: with_metadata: bool whether to include metadata
Returns: Dictionary with station information.
- to_json(*, with_metadata: bool = False, indent: int | bool | None = 4) str
Format station information as JSON.
Args: with_metadata: bool whether to include metadata indent: int or bool whether to indent the JSON
Returns: JSON string with station information.
- to_ogc_feature_collection(*, with_metadata: bool = False, **_kwargs) wetterdienst.model.result._StationsOgcFeatureCollection
Format station information as OGC feature collection.
Will be used by
.to_geojson().Args: with_metadata: bool whether to include metadata (information about the provider and producer)
Returns: Dictionary with station information as OGC feature collection.
- to_plot(**_kwargs: dict) plotly.graph_objects.Figure
Create a plotly figure from the stations DataFrame.
- _to_image(fmt: Literal[html, png, jpg, webp, svg, pdf], width: int | None = None, height: int | None = None, scale: float | None = None, **kwargs: dict) bytes | str
Create an image from the plotly figure.
This method is used by
.to_image()to create an image for stations from the plotly figure.
- class wetterdienst.model.result._ValuesItemDict
Bases:
typing_extensions.TypedDictType definition for dictionary of values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- station_id: str
None
- resolution: str
None
- dataset: str
None
- parameter: str
None
- date: str
None
- value: float | None
None
- quality: float | None
None
- class wetterdienst.model.result._ValuesDict
Bases:
typing_extensions.TypedDictType definition for dictionary of values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]
None
- stations: typing_extensions.NotRequired[list[wetterdienst.model.result._Station]]
None
- values: list[wetterdienst.model.result._ValuesItemDict]
None
- class wetterdienst.model.result._ValuesResult
Bases:
wetterdienst.io.export.ExportMixinResult class for values.
- stations: wetterdienst.model.result.StationsResult
None
- df: polars.DataFrame
None
- static _to_dict(df: polars.DataFrame) list[wetterdienst.model.result._ValuesItemDict]
Format values as dictionary.
This method is used both by
to_dict(), andto_ogc_feature_collection(), however, the latter one splits the DataFrame into multiple DataFrames by station and calls this method for each of them.
- to_dict(*, with_metadata: bool = False, with_stations: bool = False) wetterdienst.model.result._ValuesDict
Format values as dictionary.
- to_json(*, with_metadata: bool = False, with_stations: bool = False, indent: int | bool | None = 4) str
Format values as JSON.
- filter_by_date(date: str) polars.DataFrame
Filter values by date and return a new DataFrame.
- class wetterdienst.model.result._ValuesOgcFeature
Bases:
typing_extensions.TypedDictType definition for OGC feature of values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[Feature]
None
- properties: wetterdienst.model.result._OgcFeatureProperties
None
- geometry: wetterdienst.model.result._OgcFeatureGeometry
None
- values: list[wetterdienst.model.result._ValuesItemDict]
None
- class wetterdienst.model.result._ValuesOgcFeatureCollectionData
Bases:
typing_extensions.TypedDictType definition for OGC feature collection data of values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[FeatureCollection]
None
- features: list[wetterdienst.model.result._ValuesOgcFeature]
None
- class wetterdienst.model.result._ValuesOgcFeatureCollection
Bases:
typing_extensions.TypedDictType definition for OGC feature collection of values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]
None
- data: wetterdienst.model.result._ValuesOgcFeatureCollectionData
None
- class wetterdienst.model.result.ValuesResult
Bases:
wetterdienst.model.result._ValuesResultResult class for values.
- stations: wetterdienst.model.result.StationsResult
None
- values: wetterdienst.model.values.TimeseriesValues
None
- df: polars.DataFrame
None
- property df_stations: polars.DataFrame
Get DataFrame with stations.
- to_ogc_feature_collection(*, with_metadata: bool = False, **_kwargs) wetterdienst.model.result._ValuesOgcFeatureCollection
Format values as OGC feature collection.
- to_plot(**_kwargs: dict) plotly.graph_objects.Figure
Create a plotly figure from the values DataFrame.
- _to_image(fmt: Literal[html, png, jpg, webp, svg, pdf], width: int | None = None, height: int | None = None, scale: float | None = None, **kwargs: dict) bytes | str
Create an image from the plotly figure.
This method is used by
.to_image()to create an image for values from the plotly figure.
- class wetterdienst.model.result.HistoryResult
Result class for history data.
- stations: wetterdienst.model.result.StationsResult
None
- history: wetterdienst.model.history.History
None
- class wetterdienst.model.result._InterpolatedOrSummarizedOgcFeatureProperties
Bases:
typing_extensions.TypedDictType definition for OGC feature properties of interpolated or summarized values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- id: str
None
- name: str
None
- class wetterdienst.model.result._InterpolatedValuesItemDict
Bases:
typing_extensions.TypedDictType definition for dictionary of interpolated values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- station_id: str
None
- resolution: str
None
- dataset: str
None
- parameter: str
None
- date: str
None
- value: float | None
None
- distance_mean: float | None
None
- taken_station_ids: list[str]
None
- class wetterdienst.model.result._InterpolatedValuesDict
Bases:
typing_extensions.TypedDictType definition for dictionary of interpolated values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]
None
- stations: typing_extensions.NotRequired[list[wetterdienst.model.result._Station]]
None
- values: list[wetterdienst.model.result._InterpolatedValuesItemDict]
None
- class wetterdienst.model.result._InterpolatedValuesOgcFeature
Bases:
typing_extensions.TypedDictType definition for OGC feature of interpolated values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[Feature]
None
- properties: wetterdienst.model.result._InterpolatedOrSummarizedOgcFeatureProperties
None
- geometry: wetterdienst.model.result._OgcFeatureGeometry
None
- stations: list[wetterdienst.model.result._Station]
None
- values: list[wetterdienst.model.result._InterpolatedValuesItemDict]
None
- class wetterdienst.model.result._InterpolatedValuesOgcFeatureCollectionData
Bases:
typing_extensions.TypedDictType definition for OGC feature collection data of interpolated values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[FeatureCollection]
None
- features: list[wetterdienst.model.result._InterpolatedValuesOgcFeature]
None
- class wetterdienst.model.result._InterpolatedValuesOgcFeatureCollection
Bases:
typing_extensions.TypedDictType definition for OGC feature collection of interpolated values.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]
None
- data: wetterdienst.model.result._InterpolatedValuesOgcFeatureCollectionData
None
- class wetterdienst.model.result.InterpolatedValuesResult
Bases:
wetterdienst.model.result._ValuesResultResult class for interpolated values.
- stations: wetterdienst.model.result.StationsResult
None
- df: polars.DataFrame
None
- latlon: tuple[float, float]
None
- to_ogc_feature_collection(*, with_metadata: bool = False, **_kwargs) wetterdienst.model.result._InterpolatedValuesOgcFeatureCollection
Format interpolated values as OGC feature collection.
- to_plot(**_kwargs: dict) plotly.graph_objects.Figure
Create a plotly figure from the values DataFrame.
- _to_image(fmt: Literal[html, png, jpg, webp, svg, pdf], width: int | None = None, height: int | None = None, scale: float | None = None, **kwargs: dict) bytes | str
Create an image from the plotly figure.
This method is used by
.to_image()to create an image for interpolated values from the plotly figure.
- class wetterdienst.model.result._SummarizedValuesItemDict
Bases:
typing_extensions.TypedDictFormat summarized values as dictionary.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- station_id: str
None
- resolution: str
None
- dataset: str
None
- parameter: str
None
- date: str
None
- value: float | None
None
- distance: float | None
None
- taken_station_id: str | None
None
- class wetterdienst.model.result._SummarizedValuesDict
Bases:
typing_extensions.TypedDictFormat summarized values as dictionary.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]
None
- stations: typing_extensions.NotRequired[list[wetterdienst.model.result._Station]]
None
- values: list[wetterdienst.model.result._SummarizedValuesItemDict]
None
- class wetterdienst.model.result._SummarizedValuesOgcFeature
Bases:
typing_extensions.TypedDictFormat summarized values as OGC feature.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[Feature]
None
- properties: wetterdienst.model.result._InterpolatedOrSummarizedOgcFeatureProperties
None
- geometry: wetterdienst.model.result._OgcFeatureGeometry
None
- stations: list[wetterdienst.model.result._Station]
None
- values: list[wetterdienst.model.result._SummarizedValuesItemDict]
None
- class wetterdienst.model.result._SummarizedValuesOgcFeatureCollectionData
Bases:
typing_extensions.TypedDictFormat summarized values as OGC feature collection data.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- type: Literal[FeatureCollection]
None
- features: list[wetterdienst.model.result._SummarizedValuesOgcFeature]
None
- class wetterdienst.model.result._SummarizedValuesOgcFeatureCollection
Bases:
typing_extensions.TypedDictFormat summarized values as OGC feature collection.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- metadata: typing_extensions.NotRequired[wetterdienst.model.result._Metadata]
None
- data: wetterdienst.model.result._SummarizedValuesOgcFeatureCollectionData
None
- class wetterdienst.model.result.SummarizedValuesResult
Bases:
wetterdienst.model.result._ValuesResultCalculate summary of stations and parameters.
- stations: wetterdienst.model.result.StationsResult
None
- df: polars.DataFrame
None
- latlon: tuple[float, float]
None
- to_ogc_feature_collection(*, with_metadata: bool = False, **_kwargs) wetterdienst.model.result._SummarizedValuesOgcFeatureCollection
Export summarized values as OGC feature collection.
- to_plot(**_kwargs: dict) plotly.graph_objects.Figure
Create a plotly figure from the values DataFrame.
- _to_image(fmt: Literal[html, png, jpg, webp, svg, pdf], width: int | None = None, height: int | None = None, scale: float | None = None, **kwargs: dict) bytes | str
Create an image from the plotly figure.
This method is used by
.to_image()to create an image for summarized values from the plotly figure.
wetterdienst.model.metadata#
Metadata models for a provider.
Module Contents#
Classes#
|
Parameter model for a provider. |
|
Dataset model for a provider. |
|
Resolution model for a provider. |
|
Metadata model for a provider. |
|
Dataclass to hold a search for a parameter. |
Functions#
|
Build a MetadataModel from a dictionary. |
|
Parse parameters, either from string or tuple or MetadataModel or sequence of those. |
Data#
|
|
|
|
|
API#
- wetterdienst.model.metadata.log
‘getLogger(…)’
- wetterdienst.model.metadata.POSSIBLE_SEPARATORS
(‘/’, ‘.’, ‘:’)
- wetterdienst.model.metadata.DATASET_NAME_DEFAULT
‘data’
- class wetterdienst.model.metadata.ParameterModel(/, **data: Any)
Bases:
pydantic.BaseModelParameter model for a provider.
A provider declares only what it itself knows: the canonical
nameas a foreign key intoPARAMETERS, the source’s ownname_originaland the source’sunit. Theunit_typeis a property of the measured quantity rather than of the provider, so it is read from the canonical table instead of being declared – seeunit_typebelow.Initialization
Create a new model by parsing and validating input data from keyword arguments.
Raises [
ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.selfis explicitly positional-only to allowselfas a field name.- model_config
‘ConfigDict(…)’
- name: str
None
- name_original: str
None
- unit: str
None
- description: str | None
None
- dataset: pydantic.SkipValidation[DatasetModel]
‘Field(…)’
- property unit_type: wetterdienst.metadata.unit_type.UnitType
The unit type of the measured quantity, from the canonical parameter table.
Resolved on access rather than at import, so an unknown name is caught by
tests/test_api.py::test_metadata_parameter_tablerather than by every user paying a table lookup per declaration on every interpreter start.A plain property rather than a
computed_field, so it does not reappear inmodel_dump(). It is derived fromname, and re-emitting it per declaration would put back at the serialization layer the duplication this model exists to remove. Callers that want it from a dump can look the name up inPARAMETERS;discover()and the REST and CLI responses build their own dicts and report it as before.
- __eq__(other: object) bool
Compare two parameters.
- class wetterdienst.model.metadata.DatasetModel(**data: dict)
Bases:
pydantic.BaseModelDataset model for a provider.
Initialization
Initialize the dataset model.
- name: str
None
- name_original: str
None
- grouped: bool
None
- periods: list[wetterdienst.metadata.period.Period]
None
- description: str | None
None
- date_required: bool
None
- parameters: list[wetterdienst.model.metadata.ParameterModel]
None
- resolution: pydantic.SkipValidation[ResolutionModel]
‘Field(…)’
- __eq__(other: object) bool
Compare two datasets.
- __getitem__(item: str | int) wetterdienst.model.metadata.ParameterModel
Get a parameter by name.
- __getattr__(item: str) wetterdienst.model.metadata.ParameterModel
Get a parameter by name.
- __iter__() collections.abc.Iterator[wetterdienst.model.metadata.ParameterModel]
Iterate over all parameters.
- class wetterdienst.model.metadata.ResolutionModel(**data: dict)
Bases:
pydantic.BaseModelResolution model for a provider.
Initialization
Initialize the resolution model.
- name: str
None
- name_original: str
None
- value: wetterdienst.metadata.resolution.Resolution
‘Field(…)’
- periods: list[wetterdienst.metadata.period.Period] | None
None
- description: str | None
None
- date_required: bool | None
None
- datasets: list[wetterdienst.model.metadata.DatasetModel]
None
- classmethod validate_datasets(v: list[dict], validation_info: pydantic_core.core_schema.ValidationInfo) list[wetterdienst.model.metadata.DatasetModel]
Validate datasets and set resolution for each dataset.
- __getitem__(item: str | int) wetterdienst.model.metadata.DatasetModel
Get a dataset by name.
- __getattr__(item: str) wetterdienst.model.metadata.DatasetModel
Get a dataset by name.
- __iter__() collections.abc.Iterator[wetterdienst.model.metadata.DatasetModel]
Iterate over all datasets.
- class wetterdienst.model.metadata.MetadataModel(/, **data: Any)
Bases:
pydantic.BaseModelMetadata model for a provider.
Initialization
Create a new model by parsing and validating input data from keyword arguments.
Raises [
ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.selfis explicitly positional-only to allowselfas a field name.- name_short: str
None
- name_english: str
None
- name_local: str
None
- country: str
None
- copyright: str
None
- url: str
None
- kind: Literal[observation, forecast, derived]
None
- timezone: pydantic_extra_types.timezone_name.TimeZoneName
None
- timezone_data: pydantic_extra_types.timezone_name.TimeZoneName | Literal[dynamic]
None
- auth: bool
False
- resolutions: list[wetterdienst.model.metadata.ResolutionModel]
None
- __getitem__(item: str | int) wetterdienst.model.metadata.ResolutionModel
Get a resolution by name.
- __getattr__(item: str) wetterdienst.model.metadata.ResolutionModel
Get a resolution by name.
Alternatively, this still finds any other attribute that is not a resolution.
- __iter__() collections.abc.Iterator[wetterdienst.model.metadata.ResolutionModel]
Iterate over all resolutions.
- search_parameter(parameter_search: wetterdienst.model.metadata.ParameterSearch) list[wetterdienst.model.metadata.ParameterModel]
Search for a parameter in the metadata.
- wetterdienst.model.metadata.build_metadata_model(metadata: dict, name: str) wetterdienst.model.metadata.MetadataModel
Build a MetadataModel from a dictionary.
Attaches the descriptions kept in
metadata.source_descriptions, for parameters, datasets and resolutions alike. Those are the curated descriptions the provider docs tables have always carried. A description a provider module already declares wins, since that is a transcription of the source’s own wording and is only kept where it says at least as much as the curated text.DERIVED_DESCRIPTIONSfills only what no source supplies at all.
- class wetterdienst.model.metadata.ParameterSearch
Dataclass to hold a search for a parameter.
- resolution: str
None
- dataset: str
None
- parameter: str | None
None
- classmethod parse(value: str | collections.abc.Iterable[str] | wetterdienst.model.metadata.DatasetModel | wetterdienst.model.metadata.ParameterModel) wetterdienst.model.metadata.ParameterSearch
Parse a string or tuple or DatasetModel or ParameterModel into a ParameterSearch object.
- concat() str
Concatenate resolution, dataset and parameter with ‘/’.
- wetterdienst.model.metadata.parse_parameters(parameters: wetterdienst.model.request._PARAMETER_TYPE, metadata: wetterdienst.model.metadata.MetadataModel) list[wetterdienst.model.metadata.ParameterModel]
Parse parameters, either from string or tuple or MetadataModel or sequence of those.
wetterdienst.io.export#
Export data to various formats.
Module Contents#
Classes#
|
Postprocessing data. |
Functions#
|
Convert all datetime columns to ISO format. |
Data#
|
API#
- wetterdienst.io.export.log
‘getLogger(…)’
- class wetterdienst.io.export.ExportMixin
Postprocessing data.
This aids in collecting, filtering, formatting and emitting data acquired through the core machinery.
- df: polars.DataFrame
None
- filter_by_sql(sql: str) polars.DataFrame
Filter df using an SQL query WHERE clause.
- abstractmethod to_dict(*args: Any, **kwargs: Any) dict
Convert station information into dictionary format.
- abstractmethod to_json(*args: Any, **kwargs: Any) str
Convert station information into JSON format.
- abstractmethod to_ogc_feature_collection(*args: Any, with_metadata: bool, **kwargs: Any) dict
Convert station information into OGC Feature Collection format.
Abstract method implementation.
- to_geojson(*, with_metadata: bool = False, indent: int | bool | None = 4, **_kwargs: Any) str
Convert station information into GeoJSON format.
Args: with_metadata: Include metadata in GeoJSON indent: Indentation level for JSON output
Returns: GeoJSON string
- to_csv(**kwargs: Any) str
Convert DataFrame to CSV format.
Args: **kwargs: Additional arguments passed to the CSV writer
Returns: CSV string
- abstractmethod to_plot(**kwargs: Any) plotly.graph_objs.Figure
Create a plotly figure from the DataFrame.
- abstractmethod _to_image(**kwargs: Any) bytes | str
Create an image from the plotly figure.
- to_image(**kwargs: Any) bytes | str
Create an image from the plotly figure.
Args: **kwargs: Additional arguments passed to the image creation method
Returns: Image data as bytes or string
- to_format(fmt: str, **kwargs: Any) str | bytes
Format data according to the specified format.
The formatting is done by one of the following methods:
to_jsonto_csvto_geojsonto_image
Args: fmt: Output format **kwargs: Additional arguments passed to the formatting method
Returns: Formatted data
- static _filter_by_sql(df: polars.DataFrame, sql: str) polars.DataFrame
Filter df using an SQL query WHERE clause.
This implementation is based on DuckDB, so please have a look at its SQL documentation.
https://duckdb.org/docs/sql/introduction
Args: df: DataFrame to filter sql: SQL WHERE clause
Returns: Filtered DataFrame
- to_target(target: str, if_exists: Literal[replace, append, fail, skip] = 'replace') None
Emit data to a target.
The target is identified by a connection string.
Examples:
duckdb://dwd.duckdb?table=weather
influxdb://localhost/?database=dwd&table=weather
crate://localhost/?database=dwd&table=weather
Dispatch data to different data sinks. Currently, SQLite, DuckDB, InfluxDB and CrateDB are implemented. However, through the SQLAlchemy layer, it should actually work with any supported SQL database.
https://docs.sqlalchemy.org/en/13/dialects/
Args: target: Connection string if_exists: Behavior when the target already exists. Options: ‘replace’, ‘append’, ‘fail’, ‘skip’. Default is ‘replace’. - ‘replace’: Drop and recreate the target (default, backward compatible) - ‘append’: Append data to the target - ‘fail’: Raise error if target exists - ‘skip’: Do not write if target exists (only for supported backends)
Raises: KeyError: Unknown export
Returns: None (data is emitted to the target)
- wetterdienst.io.export.convert_datetimes(df: polars.DataFrame) polars.DataFrame
Convert all datetime columns to ISO format.
wetterdienst.settings#
Settings for the wetterdienst package.
Module Contents#
Classes#
|
Authentication credentials for providers requiring API keys. |
|
Settings for the wetterdienst package. |
Functions#
|
Build the per-parameter search radius from the canonical parameter table. |
Data#
|
|
|
|
|
|
|
API#
- wetterdienst.settings.log
‘getLogger(…)’
- wetterdienst.settings._UNIT_CONVERTER_TARGETS
‘keys(…)’
- class wetterdienst.settings.Auth(/, **data: Any)
Bases:
pydantic.BaseModelAuthentication credentials for providers requiring API keys.
Initialization
Create a new model by parsing and validating input data from keyword arguments.
Raises [
ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.selfis explicitly positional-only to allowselfas a field name.- aemet: str | None
‘Field(…)’
- knmi: str | None
‘Field(…)’
- metno_frost: tuple[str, str] | None
‘Field(…)’
- ceda: tuple[str, str] | None
‘Field(…)’
- classmethod validate_metno_frost(value: tuple[str, str] | str | None) tuple[str, str] | None
- classmethod validate_ceda(value: tuple[str, str] | str | None) tuple[str, str] | None
Parse the CEDA (username, password) pair, e.g. from
WD_AUTH__CEDA=username:password.
- wetterdienst.settings._STATION_DISTANCE_HOMOGENEOUS
40.0
- wetterdienst.settings._STATION_DISTANCE_HETEROGENEOUS
20.0
- wetterdienst.settings._default_geo_station_distance() collections.defaultdict[str, float]
Build the per-parameter search radius from the canonical parameter table.
Which names get the shorter radius used to be written out here, a copy of a classification the table already holds. Only those names are put in the dict; the default factory answers for every other parameter, so the setting a user sees and overrides stays the short list of exceptions rather than all 514 names.
- class wetterdienst.settings.Settings(_case_sensitive: bool | None = None, _nested_model_default_partial_update: bool | None = None, _env_prefix: str | None = None, _env_prefix_target: pydantic_settings.sources.EnvPrefixTarget | None = None, _env_file: pydantic_settings.sources.DotenvType | None = ENV_FILE_SENTINEL, _env_file_encoding: str | None = None, _env_ignore_empty: bool | None = None, _env_nested_delimiter: str | None = None, _env_nested_max_split: int | None = None, _env_parse_none_str: str | None = None, _env_parse_enums: bool | None = None, _cli_prog_name: str | None = None, _cli_parse_args: bool | list[str] | tuple[str, ...] | None = None, _cli_settings_source: pydantic_settings.sources.CliSettingsSource[Any] | None = None, _cli_parse_none_str: str | None = None, _cli_hide_none_type: bool | None = None, _cli_avoid_json: bool | None = None, _cli_enforce_required: bool | None = None, _cli_use_class_docs_for_groups: bool | None = None, _cli_show_env_vars: bool | None = None, _cli_exit_on_error: bool | None = None, _cli_prefix: str | None = None, _cli_flag_prefix_char: str | None = None, _cli_implicit_flags: bool | Literal[dual, toggle] | None = None, _cli_ignore_unknown_args: bool | None = None, _cli_kebab_case: bool | Literal[all, no_enums] | None = None, _cli_shortcuts: collections.abc.Mapping[str, str | list[str]] | None = None, _secrets_dir: pydantic_settings.sources.PathType | None = None, _build_sources: tuple[tuple[pydantic_settings.sources.PydanticBaseSettingsSource, ...], dict[str, Any]] | None = None, **values: Any)
Bases:
pydantic_settings.BaseSettingsSettings for the wetterdienst package.
Initialization
Create a new model by parsing and validating input data from keyword arguments.
Raises [
ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.selfis explicitly positional-only to allowselfas a field name.- model_config
‘SettingsConfigDict(…)’
- cache_disable: bool
‘Field(…)’
- cache_dir: pathlib.Path
‘Field(…)’
- fsspec_client_kwargs: dict
‘Field(…)’
- auth: wetterdienst.settings.Auth
‘Field(…)’
- use_certifi: bool
‘Field(…)’
- read_bufr: bool
‘Field(…)’
- ts_humanize: bool
True
- ts_shape: Literal[wide, long]
‘long’
- ts_convert_units: bool
True
- ts_unit_targets: dict[str, str]
‘Field(…)’
- ts_skip_empty: bool
False
- ts_skip_threshold: float
0.95
- ts_skip_criteria: Literal[min, mean, max]
‘min’
- ts_complete: bool
False
- ts_drop_nulls: bool
True
- ts_geo_station_distance: collections.defaultdict[str, float]
‘Field(…)’
- ts_geo_use_nearby_station_distance: Annotated[float, Field(strict=True, ge=0)] | None
1.0
- ts_geo_min_gain_of_value_pairs: Annotated[float, Field(strict=True, ge=0)]
0.1
- ts_geo_num_additional_stations: Annotated[int, Field(strict=True, ge=0)]
3
- classmethod validate_ts_unit_targets_before(values: dict[str, str] | None) dict[str, str]
Validate the unit targets.
- classmethod validate_ts_unit_targets_after(values: dict[str, str]) dict[str, str]
Validate the unit targets.
- classmethod validate_ts_geo_station_distance(values: dict[str, float] | None) dict[str, float]
Validate the interpolation station distance settings.
- property ts_tidy: bool
Return whether the time series is in tidy format.
- validate() wetterdienst.settings.Settings
Validate the settings.
- __repr__() str
Return the settings as a JSON string.
- __str__() str
Return the settings as a string.
wetterdienst.util.geo#
Geo utilities for the wetterdienst package.
Module Contents#
Functions#
|
Obtain the nearest neighbours using a simple distance computation. |
|
Convert degree minutes (floats) to decimal degree. |
|
Convert degree minutes seconds (string) to decimal degree. |
Data#
|
|
|
API#
- wetterdienst.util.geo.pc: Any
None
- wetterdienst.util.geo.EARTH_RADIUS_IN_KM
6371
- wetterdienst.util.geo.derive_nearest_neighbours(latitudes: pyarrow.Array, longitudes: pyarrow.Array, q_lat: float, q_lon: float) list[float]
Obtain the nearest neighbours using a simple distance computation.
Args: latitudes: latitudes in degree longitudes: longitudes in degree q_lat: latitude of the query point q_lon: longitude of the query point
Returns: Tuple of distances and ranks of nearest to most distant station
- wetterdienst.util.geo.convert_dm_to_dd(dm: polars.Series) polars.Series
Convert degree minutes (floats) to decimal degree.
Args: dm: Series with degree minutes as float
Returns: Series with decimal degree
- wetterdienst.util.geo.convert_dms_string_to_dd(dms: polars.Series) polars.Series
Convert degree minutes seconds (string) to decimal degree.
Args: dms: Series with degree minutes seconds as string
Returns: Series with decimal degree
wetterdienst.util.network#
Network utilities for the wetterdienst package.
Module Contents#
Classes#
|
File object for the network utilities. |
|
File-based cache for FSSPEC. |
|
HTTPFileSystem with cache support. |
|
Manage multiple FSSPEC instances keyed by cache expiration time. |
Functions#
|
Create an SSL context optionally using certifi certificates. |
|
Create a listing of all files of a given path on the server. |
|
List the immediate contents (files and subdirectories) of a given path on the server, non-recursively. |
|
Download a specified file from the server. |
|
Download multiple files from the server concurrently. |
Data#
|
API#
- wetterdienst.util.network.log
‘getLogger(…)’
- wetterdienst.util.network._create_ssl_context(*, use_certifi: bool) ssl.SSLContext | None
Create an SSL context optionally using certifi certificates.
Args: use_certifi: If True, use certifi certificate bundle instead of system certificates.
Returns: An SSL context configured with certifi certificates if requested, None otherwise.
- class wetterdienst.util.network.File
File object for the network utilities.
- url: str
None
The URL of the file.
- property filename: str
The filename of the file.
- content: io.BytesIO | Exception
None
The content of the file as a BytesIO object.
- status: int
None
The status code of the file download, if available.
- raise_if_exception() None
Raise an exception if the content is not a BytesIO object.
For NoInternetError, logs at debug level and returns silently instead of raising, allowing callers to return empty frames rather than propagating the error.
- property is_no_internet_error: bool
Check if the content is a NoInternetError.
- property nbytes: int
Return the number of bytes in the file content.
- property is_empty: bool
Check if the file content is empty.
- class wetterdienst.util.network.FileDirCache(listings_expiry_time: float, *, use_listings_cache: bool, listings_cache_location: pathlib.Path | None = None)
Bases:
collections.abc.MutableMappingFile-based cache for FSSPEC.
Initialization
Initialize the FileDirCache.
Args: listings_expiry_time: Time in seconds that a listing is considered valid. use_listings_cache: If False, this cache never returns items, but always reports KeyError. listings_cache_location: Directory path at which the listings cache file is stored.
- __getitem__(item: str) io.BytesIO
Draw item as fileobject from cache, retry if timeout occurs.
- clear() None
Clear cache.
- __len__() int
Return number of items in cache.
- __contains__(item: object) bool
Check if item is in cache and not expired.
- __setitem__(key: str, value: io.BytesIO) None
Store fileobject in cache.
- __delitem__(key: str) None
Remove item from cache.
- __iter__() collections.abc.Iterator[str]
Iterate over keys in cache.
- __reduce__() tuple
Return state information for pickling.
- class wetterdienst.util.network.HTTPFileSystem(/, *, use_listings_cache: bool, listings_expiry_time: float, listings_cache_location: pathlib.Path | None = None, use_certifi: bool = False, **kwargs)
Bases:
fsspec.implementations.http.HTTPFileSystemHTTPFileSystem with cache support.
Initialization
Initialize the HTTPFileSystem.
Args: use_listings_cache: If False, this cache never returns items, but always reports KeyError, listings_expiry_time: Time in seconds that a listing is considered valid. If None, listings_cache_location: Directory path at which the listings cache file is stored. If None, use_certifi: If True, use certifi certificate bundle instead of system certificates. *args: Additional arguments. **kwargs: Additional keyword arguments.
- class wetterdienst.util.network.NetworkFilesystemManager
Manage multiple FSSPEC instances keyed by cache expiration time.
Each thread gets its own set of filesystem instances to avoid thread-safety issues with WholeFileCacheFileSystem’s in-memory metadata cache.
- _thread_local: ClassVar[threading.local]
‘local(…)’
- classmethod _get_filesystems() dict[str, wetterdienst.util.network.HTTPFileSystem | fsspec.implementations.cached.WholeFileCacheFileSystem]
Return the per-thread filesystem registry.
- static _client_kwargs_suffix(client_kwargs: dict | None) str
Return a short stable hash suffix that distinguishes different client_kwargs (e.g. auth headers).
- static resolve_ttl(cache_expiry: wetterdienst.metadata.cache.CacheExpiry) tuple[str, float | int | Literal[False]]
Resolve the cache expiration time.
Args: cache_expiry: The cache expiration time.
Returns: The cache expiration time as name and value.
- classmethod register(cache_dir: pathlib.Path, cache_expiry: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.NO_CACHE, client_kwargs: dict | None = None, *, cache_disable: bool, use_certifi: bool = False) None
Register a new filesystem instance for a given cache expiration time.
Args: cache_dir: The cache directory to use for the filesystem. cache_expiry: The cache expiration time. client_kwargs: Additional keyword arguments for the client. cache_disable: If True, the cache is disabled. use_certifi: If True, use certifi certificate bundle instead of system certificates.
Returns: None
- classmethod get(cache_dir: pathlib.Path, cache_expiry: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.NO_CACHE, client_kwargs: dict | None = None, *, cache_disable: bool, use_certifi: bool = False) wetterdienst.util.network.HTTPFileSystem | fsspec.implementations.cached.WholeFileCacheFileSystem
Get a filesystem instance for a given cache expiration time.
Args: cache_dir: The cache directory to use for the filesystem. cache_expiry: The cache expiration time. client_kwargs: Additional keyword arguments for the client. cache_disable: If True, the cache is disabled use_certifi: If True, use certifi certificate bundle instead of system certificates.
Returns: The filesystem instance.
- wetterdienst.util.network.list_remote_files_fsspec(url: str, settings: wetterdienst.settings.Settings, cache_expiry: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.FILEINDEX) list[str]
Create a listing of all files of a given path on the server.
The default ttl with
CacheExpiry.FILEINDEXis “5 minutes”.Args: url: The URL to list files from. settings: The settings to use for the listing. cache_expiry: The cache expiration time.
Returns: A list of all files on the server
- wetterdienst.util.network.list_remote_directory_fsspec(url: str, settings: wetterdienst.settings.Settings, cache_expiry: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.FILEINDEX) list[dict]
List the immediate contents (files and subdirectories) of a given path on the server, non-recursively.
Unlike
list_remote_files_fsspec, this does not descend into subdirectories, which is useful for servers exposing a deeply nested directory tree where the folder names themselves carry enough information (e.g. a date range) to decide which subdirectories are actually worth descending into.Args: url: The URL to list the contents of. settings: The settings to use for the listing. cache_expiry: The cache expiration time.
Returns: A list of fsspec detail dicts (with “name” and “type” keys, among others) for each entry.
- wetterdienst.util.network.download_file(url: str, cache_dir: pathlib.Path, ttl: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.NO_CACHE, client_kwargs: dict | None = None, *, cache_disable: bool = False, use_certifi: bool = False) wetterdienst.util.network.File
Download a specified file from the server.
Args: url: The URL of the file to download. cache_dir: The cache directory to use for the filesystem. ttl: The cache expiration time. client_kwargs: Additional keyword arguments for the client. cache_disable: If True, the cache is disabled. use_certifi: If True, use certifi certificate bundle instead of system certificates.
Returns: A BytesIO object containing the downloaded file.
- wetterdienst.util.network.download_files(urls: list[str], cache_dir: pathlib.Path, ttl: wetterdienst.metadata.cache.CacheExpiry = CacheExpiry.NO_CACHE, client_kwargs: dict | None = None, *, cache_disable: bool = False, use_certifi: bool = False) list[wetterdienst.util.network.File]
Download multiple files from the server concurrently.