Release Notes

v0.3.0 (main)

  • Bug fix in custom static and streaming quality control tests to use a specific column

  • Minor updates to address pandas and yaml depreciation warnings

  • Updated test to use pytest instead of nose

  • Added GitHub Actions

  • Added Python 3.9 and 3.10 tests, dropped 3.6 and 3.7 tests

v0.2.0 (March 5, 2021)

  • Replaced the use of Excel files in examples/tests with CSV files. The Excel files were causing test failures.

  • Added min_failures to the streaming outlier test

  • Replaced mutable default arguments with None

  • Removed pecos logo from monitoring reports

  • Added timestamp to logger

  • Updated documentation and tests

v0.1.9 (November 2, 2020)

  • Added the ability to use custom quality control test functions in static or streaming analysis. The methods, check_custom_static and check_custom_streaming, allow the user to supply a custom function that is used to determine if data is anomalous. The custom tests also allow the user to return metadata that contains information about the quality control test.

    • The streaming analysis loops through the data using a moving window to determine if data point is normal or anomalous. If the data point is deemed anomalous it is omitted from the history and not used to determine the status of subsequent data points.

    • The static analysis operates on the entire data set, and while it can include operations like moving windows, it does not update the history based on the test results.

  • The following input arguments were changed or added:

    • In check_outlier, the input argument window was changed to None (not used), absolute value was changed to False, and an input argument streaming was added to use streaming analysis (default value is False). Changed the order of key and window to be more consistent with other quality control tests.

    • In check_delta, the input argument window is no longer optional

  • Added property data to the PerformanceMonitoring class. pm.data is equivalent to pm.df (pm.df was retained for backward compatibility)

  • Added the ability to create monitoring reports using a LaTeX template. Small changes in the HTML template.

  • Added the option to specify a date format string to timeseries plots.

  • Fixed a bug in the way masks are generated. Data points that have Null values were being assigned to False, indicating that a quality control test failed. Null values are now assumed to be True, unless a specific test fails (e.g. check_missing).

  • Updated the boolean mask used in the code to have a consistent definition (True = data point pass all tests, False = data point did not pass at least one test.)

  • Added an example in the docs to illustrate analysis of continuous data

  • Added Python 3.8 tests

  • Updated documentation and tests

v0.1.8 (January 9, 2020)

  • Added properties to the PerformanceMonitoring object to return the following:

    • Boolean mask, pm.mask. Indicates data that failed a quality control test. This replaces the method ``get_test_results_mask``(API change).

    • Cleaned data, pm.cleaned_data. Data that failed a quality control test are replaced by NaN.

  • Added the ability to run quality control tests as individual functions. These functions allow the user to use Pecos without creating a PerformanceMonitoring object. Each function returns cleaned data, a boolean mask, and a summary of quality control test results.

  • io and graphics functions were updated to use specific components of the PerformanceMonitoring class (instead of requiring an instance of the class). This changes the API for write_monitoring_report, write_dashboard, and plot_test_results.

  • Filenames are now an optional parameter in io and graphics functions, this changes the API for write_metrics, write_test_results, and plot_test_results.

  • Updated metrics:

    • Added time_derivative which returns a derivative time series for each column of data

    • qci, rmse, time_integral, probability_of_detection, and false_alarm_rate now return 1 value per column of data (API change)

    • pv metrics were also updated to return 1 value per column (API change)

    • Deprecated per_day option. Data can be grouped by custom time intervals before computing metrics (API change)

  • Efficiency improvements to check_delta. As part of these changes, the optional input argument absolute_value has been removed and direction has been added (API change). If direction is set to positive, then the test only identify positive deltas (the min occurs before the max). If direction is set to negative, then the test only identify negative deltas (the max occurs before the min).

  • Timestamp indexes down to millisecond resolution are supported

  • Added additional helper functions in pecos.utils to convert to/from datetime indexes. Methods get_elapsed_time and get_clock_time were removed from the PerformanceMonitoring class (API change).

  • Moved functionality to evaluate strings from the PerformanceMonitoring class into a stand alone utility function (API change).

  • Removed option to smooth data using a rolling mean within the quality control tests (API change). Preprocessing steps should be done before the quality control test is run.

  • Added Python 3.7 tests, dropped Python 2.7 and 3.5 tests

  • Updated examples, tests, and documentation

v0.1.7 (June 2, 2018)

  • Added quality control test to identify outliers.

  • Bug fix to allow for sub-second data frequency.

  • Dropped ‘System Name’ from the analysis and test results, this added assumptions about column names in the code.

  • Changed ‘Start Date’ and ‘End Date’ to ‘Start Time’ and ‘End Time’ in the test results.

  • New data added to a PerformanceMonitoring object using add_dataframe now overrides existing data if there are shared indexes and columns.

  • Removed add_signal method, use add_dataframe instead.

  • Adding a translation dictionary to the analysis is now optional. A 1:1 map of column names is generated when data is added to the PerformanceMonitoring object using add_dataframe.

  • Added Python 3.6 tests.

  • Removed Python 3.4 tests (Pandas dropped support for Python 3.4 with version 0.21.0).

  • Updates to check_range requires Pandas 0.23.0.

  • Updated documentation, added doctests.

v0.1.6 (August 14, 2017)

  • Added readme and license file to manifest to fix pip install

v0.1.5 (June 23, 2017)

  • Added ability to check for regular or irregular timestamps in check_timestamp.

  • Added probability of detection and false alarm metrics.

  • Added check_delta method to check bounds on the difference between max and min data values within a rolling window.

  • Added graphics method to create interactive graphics using plotly.

  • Added graphics method to create day-of-year heatmaps.

  • Method named plot_colorblock changed to plot_heatmap (API change).

  • Added data acquisition method to transfer data from sensors to an SQL database.

  • Added dashboard example that uses Pandas Styling to color code tables based on values.

  • Added graphics tests.

  • Updated documentation.

v0.1.4 (December 15, 2016)

Some of the changes in this release are not backward compatible:

  • Added capability to allow multiple html links in dashboards (API change).

  • Updated send_email function to use smptlib (API change).

  • Added additional options in html reports to set figure size and image width.

  • Bug fix setting axis limits in figures.

  • Bug fix for reporting duplicate time steps.

  • Improved efficiency for get_clock_time function.

  • Added dashboard example that uses color blocks to indicate number of test failures.

  • Removed basic_pvlib_performance_model, the pv_example now uses pvlib directly to compute a basic model (API change).

v0.1.3 (August 2, 2016)

This is a minor release, changes include:

  • Bug fix for DataFrames using timezones. There was an issue retaining the timezone across the entire pecos analysis chain. The timezone was not stored properly in the testing results. This is a known pandas bug. The fix in Pecos includes stronger tests for analysis that use timezones.

  • The use of Jinja for html report templates

  • Cleaned up examples

v0.1.2 (June 6, 2016)

This is a minor release, changes include:

  • Minor changes to the software to support Python 3.4 and 3.5

  • Default image format changed from jpg to png

  • Datatables format options added to dashboards

  • Additional testing

v0.1.1 (May 6, 2016)

This is a minor release, changes include:

  • Added a pv module, includes basic methods to compute energy, insolation, performance ratio, performance index, energy yield, clearness index, and a basic pv performance model.

  • Added method to compute time integral and RMSE to metrics module

  • Cleaned up examples, API, and documentation

  • Software test harness run through TravisCI and analyzed using Coveralls

  • Documentation hosted on readthedocs

v0.1.0 (March 31, 2016)

This is the first official release of Pecos. Features include:

  • PerformanceMonitoring class used to run quality control tests and store results. The class includes the ability to add time filters and translation dictionaries. Quality control tests include checks for timestamp, missing and corrupt data, data out of range, and increment data out of range.

  • Quality control index used to quantify test failures

  • HTML report templates for monitoring reports and dashboards

  • Graphics capabilities

  • Basic tutorials

  • Preliminary software test harness, run using nosetests

  • Basic user manual including API documentation