pecos.io module¶

The io module contains functions to read/send data and write results to files/html reports.

pecos.io.read_campbell_scientific(filename, index_col='TIMESTAMP', encoding=None)[source]¶

Read Campbell Scientific CSV file.

Parameters:

filename (string) – File name
index_col (string, optional) – Index column name, default = ‘TIMESTAMP’
encoding (string, optional) – Character encoding (i.e. utf-16)

Returns:

pandas DataFrame – Data

pecos.io.send_email(subject, body, recipient, sender, attachment=None, host='localhost', username=None, password=None)[source]¶

Send email using Python smtplib and email packages.

Parameters:

subject (string) – Subject text
body (string) – Email body, in HTML or plain format
recipient (list of string) – Recipient email address or addresses
sender (string) – Sender email address
attachment (string, optional) – Name of file to attach
host (string, optional) – Name of email host (or host:port), default = ‘localhost’
username (string, optional) – Email username for authentication
password (string, optional) – Email password for authentication

pecos.io.write_metrics(metrics, filename='metrics.csv')[source]¶

Write metrics file.

Parameters:

metrics (pandas DataFrame) – Data to add to the metrics file
filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘metrics.csv’

Returns:

string – filename

pecos.io.write_test_results(test_results, filename='test_results.csv')[source]¶

Write test results file.

Parameters:

test_results (pandas DataFrame) – Summary of the quality control test results (pm.test_results)
filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘test_results.csv’

Returns:

string – filename

pecos.io.write_monitoring_report(data, test_results, test_results_graphics=None, custom_graphics=None, metrics=None, title='Pecos Monitoring Report', config=None, logo=False, im_width_test_results=1, im_width_custom=1, im_width_logo=0.1, encode=False, file_format='html', filename='monitoring_report.html')[source]¶

Generate a monitoring report. The monitoring report is used to report quality control test results for a single system. The report includes custom graphics, performance metrics, and test results.

Parameters:

data (pandas DataFrame) – Data, indexed by time (pm.data)
test_results (pandas DataFrame) – Summary of the quality control test results (pm.test_results)
test_results_graphics (list of strings or None, optional) – Graphics files, with full path. These graphics highlight data points that failed a quality control test, created using pecos.graphics.plot_test_results(). If None, test results graphics are not included in the report.
custom_graphics (list of strings or None, optional) – Custom files, with full path. Created by the user. If None, custom graphics are not included in the report.
metrics (pandas Series or DataFrame, optional) – Performance metrics to add as a table to the monitoring report
title (string, optional) – Monitoring report title, default = ‘Pecos Monitoring Report’
config (dictionary or None, optional) – Configuration options, to be printed at the end of the report. If None, configuration options are not included in the report.
logo (string, optional) – Graphic to be added to the report header
im_width_test_results (float, optional) – Image width as a fraction of page size, for test results graphics, default = 1
im_width_custom (float, optional) – Image width as a fraction of page size, for custom graphics, default = 1
im_width_logo (float, optional) – Image width as a fraction of page size, for the logo, default = 0.1
encode (boolean, optional) – Encode graphics in the html, default = False
filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘monitoring_report.html’

Returns:

string – filename

pecos.io.write_dashboard(column_names, row_names, content, title='Pecos Dashboard', footnote='', logo=False, im_width=250, datatables=False, encode=False, filename='dashboard.html')[source]¶

Generate a dashboard. The dashboard is used to compare results across multiple systems. Each cell in the dashboard includes custom system graphics and metrics.

Parameters:

column_names (list of strings) – Column names listed in the order they should appear in the dashboard, i.e. [‘location1’, ‘location2’]
row_names (list of strings) – Row names listed in the order they should appear in the dashboard, i.e. [‘system1’, ‘system2’]
content (dictionary) –
Dashboard content for each cell. Dictionary keys are tuples indicating the row name and column name, i.e. (‘row name’, ‘column name’), where ‘row name’ is in the list row_names and ‘column name’ is in the list column_names.

For each (‘row name’, ‘column name’) key, another dictionary is defined that contains the content to be included in each cell of the dashboard. Each cell can contain text, graphics, a table, and an html link. These are defined using the following case sensitive keys:
- text (string) = text at the top of each cell
- graphics (list of strings) = a list of graphics file names. Each file name includes the full path
- table (string) = a table in html format, for example a table of performance metrics. DataFrames can be converted to an html string using df.to_html() or df.transpose().to_html(). Values in the table can be color coded using pandas Styler class.
- link (dict) = a dictionary where keys define the name of the link and values define the html link (with full path)
For example:
```
content = {('row name', 'column name'): {
    'text': 'text at the top',
    'graphic': ['C:\\pecos\\results\\custom_graphic.png'],
    'table': df.to_html(),
    'link': {'Link to monitoring report': 'C:\\pecos\\results\\monitoring_report.html'}}
```
title (string, optional) – Dashboard title, default = ‘Pecos Dashboard’
footnote (string, optional) – Text to be added to the end of the report
logo (string, optional) – Graphic to be added to the report header
im_width (float, optional) – Image width in the HTML report, default = 250
datatables (boolean, optional) – Use datatables.net to format the dashboard, default = False. See https://datatables.net/ for more information.
encode (boolean, optional) – Encode graphics in the html, default = False
filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘dashboard.html’

Returns:

string – filename

pecos.io.device_to_client(config)[source]¶

Read channels on modbus device, scale and calibrate the values, and store the data in a MySQL database. The inputs are provided by a configuration dictionary that describe general information for data acquisition and the devices.

Parameters:: config (dictionary) – Configuration options, see Data acquisition