pecos.io module

The io module contains functions to read/send data and write results to files/html reports.

pecos.io.read_campbell_scientific(filename, index_col='TIMESTAMP', encoding=None)[source]

Read Campbell Scientific CSV file.

Parameters:
  • filename (string) – File name

  • index_col (string, optional) – Index column name, default = ‘TIMESTAMP’

  • encoding (string, optional) – Character encoding (i.e. utf-16)

Returns:

pandas DataFrame – Data

pecos.io.send_email(subject, body, recipient, sender, attachment=None, host='localhost', username=None, password=None)[source]

Send email using Python smtplib and email packages.

Parameters:
  • subject (string) – Subject text

  • body (string) – Email body, in HTML or plain format

  • recipient (list of string) – Recipient email address or addresses

  • sender (string) – Sender email address

  • attachment (string, optional) – Name of file to attach

  • host (string, optional) – Name of email host (or host:port), default = ‘localhost’

  • username (string, optional) – Email username for authentication

  • password (string, optional) – Email password for authentication

pecos.io.write_metrics(metrics, filename='metrics.csv')[source]

Write metrics file.

Parameters:
  • metrics (pandas DataFrame) – Data to add to the metrics file

  • filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘metrics.csv’

Returns:

string – filename

pecos.io.write_test_results(test_results, filename='test_results.csv')[source]

Write test results file.

Parameters:
  • test_results (pandas DataFrame) – Summary of the quality control test results (pm.test_results)

  • filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘test_results.csv’

Returns:

string – filename

pecos.io.write_monitoring_report(data, test_results, test_results_graphics=None, custom_graphics=None, metrics=None, title='Pecos Monitoring Report', config=None, logo=False, im_width_test_results=1, im_width_custom=1, im_width_logo=0.1, encode=False, file_format='html', filename='monitoring_report.html')[source]

Generate a monitoring report. The monitoring report is used to report quality control test results for a single system. The report includes custom graphics, performance metrics, and test results.

Parameters:
  • data (pandas DataFrame) – Data, indexed by time (pm.data)

  • test_results (pandas DataFrame) – Summary of the quality control test results (pm.test_results)

  • test_results_graphics (list of strings or None, optional) – Graphics files, with full path. These graphics highlight data points that failed a quality control test, created using pecos.graphics.plot_test_results(). If None, test results graphics are not included in the report.

  • custom_graphics (list of strings or None, optional) – Custom files, with full path. Created by the user. If None, custom graphics are not included in the report.

  • metrics (pandas Series or DataFrame, optional) – Performance metrics to add as a table to the monitoring report

  • title (string, optional) – Monitoring report title, default = ‘Pecos Monitoring Report’

  • config (dictionary or None, optional) – Configuration options, to be printed at the end of the report. If None, configuration options are not included in the report.

  • logo (string, optional) – Graphic to be added to the report header

  • im_width_test_results (float, optional) – Image width as a fraction of page size, for test results graphics, default = 1

  • im_width_custom (float, optional) – Image width as a fraction of page size, for custom graphics, default = 1

  • im_width_logo (float, optional) – Image width as a fraction of page size, for the logo, default = 0.1

  • encode (boolean, optional) – Encode graphics in the html, default = False

  • filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘monitoring_report.html’

Returns:

string – filename

pecos.io.write_dashboard(column_names, row_names, content, title='Pecos Dashboard', footnote='', logo=False, im_width=250, datatables=False, encode=False, filename='dashboard.html')[source]

Generate a dashboard. The dashboard is used to compare results across multiple systems. Each cell in the dashboard includes custom system graphics and metrics.

Parameters:
  • column_names (list of strings) – Column names listed in the order they should appear in the dashboard, i.e. [‘location1’, ‘location2’]

  • row_names (list of strings) – Row names listed in the order they should appear in the dashboard, i.e. [‘system1’, ‘system2’]

  • content (dictionary) –

    Dashboard content for each cell. Dictionary keys are tuples indicating the row name and column name, i.e. (‘row name’, ‘column name’), where ‘row name’ is in the list row_names and ‘column name’ is in the list column_names.

    For each (‘row name’, ‘column name’) key, another dictionary is defined that contains the content to be included in each cell of the dashboard. Each cell can contain text, graphics, a table, and an html link. These are defined using the following case sensitive keys:

    • text (string) = text at the top of each cell

    • graphics (list of strings) = a list of graphics file names. Each file name includes the full path

    • table (string) = a table in html format, for example a table of performance metrics. DataFrames can be converted to an html string using df.to_html() or df.transpose().to_html(). Values in the table can be color coded using pandas Styler class.

    • link (dict) = a dictionary where keys define the name of the link and values define the html link (with full path)

    For example:

    content = {('row name', 'column name'): {
        'text': 'text at the top',
        'graphic': ['C:\\pecos\\results\\custom_graphic.png'],
        'table': df.to_html(),
        'link': {'Link to monitoring report': 'C:\\pecos\\results\\monitoring_report.html'}}
    

  • title (string, optional) – Dashboard title, default = ‘Pecos Dashboard’

  • footnote (string, optional) – Text to be added to the end of the report

  • logo (string, optional) – Graphic to be added to the report header

  • im_width (float, optional) – Image width in the HTML report, default = 250

  • datatables (boolean, optional) – Use datatables.net to format the dashboard, default = False. See https://datatables.net/ for more information.

  • encode (boolean, optional) – Encode graphics in the html, default = False

  • filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘dashboard.html’

Returns:

string – filename

pecos.io.device_to_client(config)[source]

Read channels on modbus device, scale and calibrate the values, and store the data in a MySQL database. The inputs are provided by a configuration dictionary that describe general information for data acquisition and the devices.

Parameters:

config (dictionary) – Configuration options, see Data acquisition