eidaqc.eida_report module

Evaluate test results and create summary report.

Iterates through the results of Eida inventory test and Eida availability test and computes summary statistics. These are documented in markdown

Requires pandoc to convert markdown reports into HTML and PDF. PDF conversion additionally requires a Latex installation that is compatible with pandoc.

Availability test results are presented as scatterplot on a map with each dot representing a station. For a meaningful image a mapping library is required. First choice is cartopy. However, on older systems and/or if you do not use conda basemap may be easier to install. Note though, that basemap is no longer actively developed in favor of cartopy. If you have neither package installed, station coordinates are used as x,y in cartesian coordinates, which is obviously not very helpful. If you want to choose your plotting tool, you can get the data from AvailabilityReport.loop_files().

Create report with command line:

$ eida rep <configfile>

or from script:

from eida_config import EidaTestConfig
from eida_report import EidaTestReport
config = EidaTestConfig(configfile, "report")
etr = EidaTestReport(config)
etr.make_md_report()
etr.make_pdf_report(self.repfile, pdfengine)
etr.make_html_report(self.repfile)

# or all three previous commands at once with
etr.daily_report()
class eidaqc.eida_report.AvailabilityReport(config)

Bases: eidaqc.eida_report.BaseReport

Create statistics from Eida Availability test results and summary text.

Most likely, you want to call:

AvailabilityReport(config).results2mdstr(
    figfilenames=["fig1.png", "fig2.png"])
Parameters

config (eidaqc.EidaTestConfig) – a configparser object which is created when reading the config file

The method creates two matplotlib-figures

  • AvailabilityReport.availability_map and

  • AvailabilityReport.hitplot

and a text summary in Markdown available as AvailabilityReport.mdstr

We assume that the availability test was run with the same parameters as in the config-file for the report. An object eida_availability.EidaAvailability() is created for the report.

Note

  • The plotting functions extract and provide vital informations for the statistics. A cleaner separation of functionality might be helpful.

_availplot_basemap(fig, x, y, c, mapgeo=None)
_availplot_cartopy(fig, x, y, c, mapgeo=None)
_availplot_nomap(fig, x, y, c)
add_keyword(keyw)

Store status codes for network statistics.

add_stats(cnt)

Hit count statistics, how many stations have how many hits.

dump_netstat()

Write networks status statistics in different formats.

failwords = ('NODATA', 'FRAGMENT', 'INCOMPL', 'METAFAIL', 'RESTFAIL')
loop_files()

Loop all networks and stations in file database, return availability and location.

Returns

data – shape is (n_station, 3). Columns are percentage, lat, lon. One line per station in database. Percentage represents the frequency of failed (0%) and successful (100%) data request to that station.

Return type

numpy.ndarray

Output is used for plotting

makeavailplot(outfile=None, mapgeo=None)
Parameters

  • outfile (str) – file name

  • mapgeo (dict) – same as self.mapgeometry

makehitplot(outfile=None)
mapgeometry = {'lat_ts': 45.0, 'llcrnrlat': 29.0, 'llcrnrlon': -11.0, 'projection': 'merc', 'resolution': 'l', 'urcrnrlat': 71.5, 'urcrnrlon': 50.0}
okwords = ('OK',)
parse_yearfile(fname, okcnt, failcnt)

Parse a single file in the file database and return the sum of evaluated status codes as well as a location.

parse_years(fpath)

Parse all files of a station.

results2mdstr(figfilenames=None)

Create markdown formatted string from results.

Parameters

figfilenames (list of 2 str or None) – filenames for the 2 figures. If None calls plt.show() i.e. figures are not saved for report.

total_number_of_stations()

Determine total number of stations, remove double entries and stations of excluded networks.

class eidaqc.eida_report.BaseReport(loggername='BaseReport')

Bases: object

Provides utilities to create a report.

Parameters

loggername (str ["BaseReport"]) – name of logger in error messages

_check_mdfile(mdfile)

Return meaningful error message when no file is available

_legacy_plot_save(outfile)

Original procedure to save plots.

Seems overly complicated and involves external program magic. The main utility seems to be to trim the white frame around the figures. This can be achieved internally with pyplot by setting plt.savefig(...,bbox_inches="tight").

display_pdf_report(pdffile)
dump2mdfile(mdfilename, mdstr=None)

Write string to file.

Manages also if the file already exists and/or is already open for writing.

Parameters

  • mdfilename (str) – output filename. Assigned to self.mdfile

  • mdstr (str, None) – content to write. Uses self.mdstr if None

Assigns file handler to self.fp

Warning

File is not closed!!! You need to execute self.fp.close() at some point.

make_html_report(mdfile=None, cssfile=None)

Convert markdown to html report using pandoc.

Parameters

  • mdfile (str or None) – if None we look for self.mdfile which is set after running self.dump2mdfile()

  • cssfile (str or None) – CSS-style file. If None we use the one in the package

make_pdf_report(mdfile=None, pdfengine='pdflatex')

Convert markdown report to pdf via pandoc.

Pandoc and a valid pandoc pdf-engine and the necessary latex packages must be installed.

On Ubuntu you may use e.g. the texlive-latex-extra package (info updated 10/2021):

apt install texlive-latex-extra

Smaller texlive versions do not ship all necessary latex packages.

Parameters

  • mdfile (str or None) – if None, we use self.mdfile which is set after running self.dump2mdfile()

  • pdfengine (str) – passed to pandoc flag –pdf-engine.

newpage()

Print markdown newpage command.

repprint(text='')
class eidaqc.eida_report.EidaTestReport(config)

Bases: eidaqc.eida_report.BaseReport

Create report of Eida availability test and inventory test.

daily_report(pdfengine='pdflatex')

Convenience function to create markdown, html and pdf report at once.

Parameters

pdfengine (str) – see BaseReport.make_pdf_report()

infostr()

Returns text incl. creation time and pandoc version

make_md_report()

Create markdown file of summary report on availability and inventory test.

  • Runs results2mdstr() on both reports.

  • Runs dump2mdfile() on both reports.

  • Adds own info string on creation time

  • names figure files as 'self.reportbase_fig123.png'

class eidaqc.eida_report.InventoryReport(config, stime=None)

Bases: eidaqc.eida_report.BaseReport

Evaluate and plot results of inventory test.

Parameters

  • config (eidaqc.EidaTestConfig) – a configparser object which is created when reading the config file

  • reference_networks (dict) –

  • stime (datetime, str or None) –

_get_logfiles()

Get inventory logfiles between now and starttime of report window.

  • Gets a list of inventory test log files which have names of form 'eia_datapath/eida_inentory_test/eida_invtest_log.YYYY-MM-DD'.

  • Create sublist with files for which date ending YYYY-MM-DD > starttime of report

  • Latest results are in file 'eida_invtest_log' without date extensions

_md_add_figure(figfile)

Create markdown formatted call of figure.

_mdstrbody()

Provide str with default information for report.

cumulate_failures(dfail, rfail, ddict, rdict)
parse_time(timestr)

Convert timestring into datetime object

print_failure_rates(directfail, routefail, directcnt, routecnt, noerrcnt, roclifailures, mode)
results2mdstr(respplot=None, mode='normal')
transref(missnet)
class eidaqc.eida_report.MetaResponsePlot(stime, granularity, plotname)

Bases: object

Create a plot with response info of all servers.

Called by InventoryReport

Parameters

  • stime –

  • granularity (float, int) –

  • plotname (str) –

dump()
finish_index()
getcol(srv, idx)
inc_fail_count(server)
inc_total_count(server)
makeplot()
set_time(ctime)
time_label(tm)