eidaqc.eida_config module

Manage configuration of Eida Network Tests.

Parameters for the tests and report are best set via the configuration file. We use the configparser module to do this.

This module converts the parameters in the config-file into the right data formats, such as int, float, dicts or datetime, which are then retrievable through the EidaTestConfig class. The parameters are bundled by task as dictionary, which are set as attributes. Three different tasks can be specified:

  • availability test

    which='av', attribute EidaTestConfig.avtest

  • inventory test

    which='inv', attribute EidaTestConfig.invtest

  • report

    which='rep', attribute EidaTestConfig.report

If which=None all three attributes are set. The dictionaries EidaTestConfig.avtest and EidaTestConfig.invtest can be used to instantiate the availability and inventory test class, respectively. The report classes take only the config-class.

To create a template file in current working directory use command line:

eida templ

This should create a file default_config.ini. If you want a different location and name, use:

eida templ <path/to/file>

Alternatively, you can call eidaqc.eida_config.create_default_configfile() in a Python script.

In addition to the config-file, the CSS-file for the HTML report is provided. Look for 'default_html_template.css'

The config-file should look like this:

[NETWORKS]
wanted_channels = HHZ, BHZ, EHZ, SHZ
# networks exclude from testing, e.g. temporary or non-european networks
exclude_networks = 1N, 1T, 3C, 4H, 5M, 7A, 8C, 9C, 9H, XK, XN, XT, XW, YW, YZ, Z3, ZF, ZJ, ZM, ZS, AI, AW, CK, CN, CX, GL, IO, IQ, KC, KP, MQ, NA, ND, NU, PF, WC, WI

[PROBABILITIES]
nl = 0.5

[SERVER_REFERENCE_NETWORKS]
nl = ODC
ge = GFZ
fr = RESIF
ch = ETH
gr = BGR
bw = LMU
ro = NIEP
ko = KOERI
hl = NOA
no = http://eida.geo.uib.no
ca = ICGC
iv = INGV

[ERROR LOGGING]
loglevel_console = DEBUG
loglevel_file = DEBUG
# unit of logfile rotation - choose from
# s (seconds), m (minutes), h (hours), d (days), midnight)
log_timeunit = h
# interval to rotate logfile for units s, m, h, d
log_interval = 1
# number of logfiles to be kept before oldest one gets deleted
log_backupcount = 2

[PATHS]
# location of temporary files (error logs, etc)
eia_tmp_path = /tmp
# location to store results
eia_datapath = /home/lehr/svn/EidaQualityCheck/trunk/eidaQC/testing/EidaTest_results

[Availability Test]
# select waveforms within the last days
eia_global_timespan_days = 365
# timeout for retrieving station metadata.
eia_timeout = 60
# minimum number of networks to get data before replacing cached inventory.
eia_min_num_networks = 80
# age of cached inventory file in seconds. if file is older, inventory is updated
maxcacheage = 432000
# minimum length of data for test request, in seconds
minreqlen = 60
# maximum length of data for test request, in seconds
maxreqlen = 600
# time to wait until next try if inventory update frm servers failed, in seconds
inv_update_waittime = 3600

[Inventory test]
timeout = 240
# endtime of request interval
endtime = now
# starttime or interval for request, counted backwards from t1 in seconds
starttime = 31536000
# rotate result file at 'midnight' (after 24h) or weekday 'w0-6' (0=monday)
rotate_log_at = midnight
# time at which rollover occurs (in utc)
rotate_log_at_time = 00:00:00
# number of files to keep from the past
inv_log_bckp_count = 12

[Report]
# number of days over which to request statistics for report.
eia_reqstats_timespan_days = 92
# css-style file for html report
eia_cssfile = /home/lehr/svn/EidaQualityCheck/trunk/eidaQC/testing/default_html_template.css
# timespan in days before now for which inventory test is evaluated.
inv_rep_timespan_days = 14
# path and name of report file
reportfile = EidaTest_report.md
# hours over which inventory results are averaged
granularity = 8
  • NETWORKS

    • wanted_channels: look for these channels when retrieving inventory. Therefore, the component (‘Z’) is not so important. For the availability test, channels/components are drawn from the full inventory of the selected station. Thus other channels than those given here might occur

    • exclude_networks: list of networks, that should be removed from the inventory and thus ignored for the availability test. Can be e.g. non-european, temporary or very small networks.

  • PROBABILITIES

    list networks for which reduced probability should be used during the random station selection of the availability test. List as Network-ID = probability

  • SERVER_REFERENCE_NETWORKS

    list of network-server-pairs (network-id = server-id) For each server, give a representative network, which is mainly delivered through that server.

  • ERROR LOGGING

    how to deal with the log and error messages. Configures the logging module. Note that this does not handle the test results.

    • loglevel_console: threat level for output on console, messages at lower level will be ignored. Can be DEBUG, INFO, WARNING, ERROR, CRITICAL. DEBUG is most verbose.

    • loglevel_file: same as above, but for messages written sent to temporary file.

    The following parameters handle the rotation of logging files. We use a TimedRotatingFileHandler A new log-file is started every log_interval log_timeunit

    • log_timeunit: unit of log_interval. Can be 'S' econds, 'M' inutes, 'H' ours or 'D' ays

    • log_interval: int, number of time units

    • log_backupcount: int, keep so many logfiles before deleting oldest. A datetime string is appended to the file name.

  • PATHS

    • eia_tmp_path: location for temporary files, e.g. the log files

    • eia_datapath: directory where test results and report files are written to.

  • Availability Test

    see comments

  • Inventory test

    see comments

  • Report

    see comments

class eidaqc.eida_config.EidaTestConfig(configfile, which=None)

Bases: object

Read parameters from file and prepare for use in Eida tests and report.

Parameters

  • configfile (str) – name of config-file

  • which ('inv', 'rep', 'av' or [None]) – specify task. ‘rep’ and None read all parameters, ‘inv’ and ‘av’ only those necessary for the corresponding test.

avtest

Can be used to initialize EidaAvailability(**config.avtest). not set if which='inv'.

Type

dict

invtest

Can be used to initialize EidaInventory(**config.invtest). not set if which='av'.

Type

dict

report

contains additional parameters for creation of reports

Type

dict

_split_ignoring_whitespace(s, sep=',')

Split string at sep and remove trailing/leading whitespaces around the list elements.

get_avtest()
get_avtest_dict()
get_datetime(t, t0=None)

Convert time string into UTCDateTime.

  • if t is “now”, current time is returned

  • if t is a string that can be handled by UTCDateTime, returns given datetime

  • if t is numeric and t0 as UTCDateTime is present, returns t_0 - t

get_invtest()
get_invtest_dict()
get_logging_handlers()

Retrieve logger-related settings.

Checks if 'eia_tmp_path' exists. If it doesn’t uses current working dir instead.

get_networks()
get_networks_servers()
get_paths()

Process paths section

TODO: Paths could be tested here instead by each module separately

get_report()
get_time(t, msg='')

Convert time string HH:MM:SS into UTCDateTime

eidaqc.eida_config.create_default_configfile(outfile=None)

Create a config file from default variables.

  • eia_datapath is set relative to current work dir.

  • eia_tmp_path is set to default temp dir

  • eia_default_css_file is copied to cwd or outfile dest.

eidaqc.eida_config.expandpath(p)
eidaqc.eida_config.load_css_template(outfile=None)

Retrieve CSS-file from package