Observation#

Overview#

The big treasure of the DWD is buried under a clutter of a file_server. The data you find here can reach back to 19th century and is represented by over 1000 stations in Germany according to the report referenced above. The amount of stations that cover a specific parameter may differ strongly, so don’t expect the amount of data to be that generous for all the parameters.

Available data/parameters on the file server is sorted in different time resolutions:

  • 1_minute - measured every minute

  • 5_minute - measured every 5 minutes

  • 10_minutes - measured every 10 minutes

  • hourly - measured every hour

  • subdaily - measured 3 times a day

  • daily - measured once a day

  • monthly - measured/summarized once a month

  • annual - measured/summarized once a year

Depending on the time resolution of the parameter you may find different periods that the data is offered in:

  • historical - values covering all the measured data

  • recent - recent values covering data from latest plus a certain range of historical data

  • now - current values covering only latest data

The period relates to the amount of data that is measured, so measuring a parameter every minute obviously results an a much bigger amount of data and thus smaller chunks of data are needed to lower the stress on data transfer, e.g. when updating your database you probably won’t need to stream all the historical data every day. On the other hand this will also save you a lot of time as the size relates to the processing time your machine will require.

The table below lists every (useful) parameter on the file server with its combinations of available time resolutions. In general only 1-minute and 10-minute data is offered in the “now” period, although this may change in the future.

The two parameter strings reflect on how we call a parameter e.g. “PRECIPITATION” and how the DWD calls the parameter e.g. “precipitation”.

This table and subsets of it can be printed with a function call of .discover() as described in the API section. Furthermore individual parameters can be queried.

Parameter details#

Cloud types#

Cloud type

Code

Cirrus

0

Cirrocumulus

1

Cirrostratus

2

Altocumulus

3

Altostratus

4

Nimbostratus

5

Stratocumulus

6

Stratus

7

Cumulus

8

Cumulonimbus

9

Automated

-1

Tidy parameters#

The tidy version of DWD observation data excludes several parameters which contain strings. Those parameters are:

  • cloud type abbreviations (1 - 4) in hourly cloud type dataset

  • total cloud cover indicator in in hourly cloudiness dataset

  • true local time in hourly solar dataset

  • visibility indicator in hourly visibility dataset

Quality#

The DWD designates its data points with specific quality levels expressed as “quality bytes”.

  • The “recent” data have not completed quality control yet.

  • The “historical” data are quality controlled measurements and observations.

The following information has been taken from PDF documents on the DWD open data server like data set description for historical hourly station observations of precipitation for Germany. Wetterdienst provides convenient access to the relevant details by using routines to parse specific sections of the PDF documents.

For example, use commands like these for accessing this information:

# Historical hourly station observations of precipitation for Germany.
# English language.
wetterdienst dwd about fields --parameter=precipitation --resolution=hourly --period=historical

# Historical 10-minute station observations of pressure, air temperature (at 5cm and 2m height), humidity and dew point for Germany.
# German language.
wetterdienst dwd about fields --parameter=air_temperature --resolution=10_minutes --period=historical --language=de

or have a look at the example program dwd_describe_fields.py.

Details#

Considerations of quality assurance are explained in Kaspar et al., 2013.

Several steps of quality control, including automatic tests for completeness, temporal and internal consistency, and against statistical thresholds based on the software QualiMet (see Spengler, 2002) and manual inspection had been applied.

Data are provided “as observed”, no homogenization has been carried out.

The history of instrumental design, observation practice, and possibly changing representativity has to be considered for the individual stations when interpreting changes in the statistical properties of the time series. It is strongly suggested to investigate the records of the station history which are provided together with the data. Note that in the 1990s many stations had the transition from manual to automated stations, entailing possible changes in certain statistical properties.

When data from both directories “historical” and “recent” are used together, the difference in the quality control procedure should be considered. There are still issues to be discovered in the historical data. The DWD welcomes any hints to improve the data basis (see contact).

Examples#

As an example, these sections display different means of quality designations related to daily/hourly and 10_minutes resolutions/products.

The quality levels “Qualitätsniveau” (QN) given here apply for the respective following columns. The values are the minima of the QN of the respective daily values. QN denotes the method of quality control, with which erroneous values are identified and apply for the whole set of parameters at a certain time.

For the individual parameters there exist quality bytes in the internal DWD data base, which are not published here. Values identified as wrong are not published.

Various methods of quality control (at different levels) are employed to decide which value is identified as wrong. In the past, different procedures have been employed. The quality procedures are coded as following.

Quality level (column header: QN_):

1- Only formal control during decoding and import
2- Controlled with individually defined criteria
3- ROUTINE control with QUALIMET and QCSY
5- Historic, subjective procedures
7- ROUTINE control, not yet corrected
8- Quality control outside ROUTINE
9- ROUTINE control, not all parameters corrected
10- ROUTINE control finished, respective corrections finished

The quality level “Qualitätsniveau” (QN) given here applies for the following columns. QN describes the method of quality control applied to a complete set of parameters, reported at a common time.

The individual parameters of the set are connected with individual quality bytes in the DWD data base, which are not given here. Values marked as wrong are not given here.

Different quality control procedures (and at different levels) have been applied to detect which values are identified as erroneous or suspicious. Over time, these procedures have changed.

Quality level (column header: QN):

1- Only formal control during decoding and import
2- Controlled with individually defined criteria
3- ROUTINE automatic control and correction with QUALIMET

Structure#