User guide¶

This guide explains the main concepts behind QUENDS and the recommended workflow for each part of the library. It is organised to mirror the package layout, so each topic below corresponds to a module in the API Reference.

For complete executable examples, see the Gallery of examples.

QUENDS analyses follow one pipeline:

raw file/array  ->  DataStream  ->  trim (steady state)  ->  statistics + uncertainty
                                         |
                  many runs  ->  Ensemble  ->  ensemble estimate

Loading data¶

The preprocessing layer converts raw simulation output into a DataStream. Each loader takes a source and one variable name, and returns a DataStream holding just the time column and that signal:

import quends as qnds

ds = qnds.from_csv("output.csv", "Q_D/Q_GBD")    # CSV
ds = qnds.from_gx("run.nc", "HeatFlux_st")         # GX NetCDF
ds = qnds.from_netcdf("data.nc", "flux")           # generic NetCDF

The time column is detected automatically (by name, then by monotonicity). Inspect what you loaded with ds.variables() (returns ['time', '<signal>']) and reach the underlying pandas DataFrame via ds.data. To analyse several signals from one file, load each into its own DataStream.

Trimming & steady state¶

Most simulation traces begin with a transient that must be removed before statistics are meaningful. DataStream.trim detects the start of the steady-state region and returns a new, trimmed DataStream:

trimmed = ds.trim(
    column_name="Q_D/Q_GBD",
    method="std",          # detection method (see below)
    window_size=10,
    start_time=0.0,        # ignore everything before this time
    robust=True,           # median + MAD instead of mean + std
)

Each method string maps to a concrete trim-strategy class in quends.base.trim (the canonical mapping lives in build_trim_strategy). Pass the method string to DataStream.trim / Ensemble.trim, or use the class directly through TrimDataStreamOperation:

start_time acts as a hard lower bound (everything before it is discarded); the detector then searches from there. The trimmed result records where it cut in trimmed.trim_metadata["sss_start"].

Statistics & uncertainty¶

Once trimmed, compute statistics with an uncertainty estimate that accounts for autocorrelation:

stats = trimmed.compute_statistics(method="non-overlapping")

The result is a StatsResult (a dict subclass with a .metadata attribute) keyed by column. Each entry includes the mean, the mean_uncertainty (standard error of the mean corrected for the effective sample size), the confidence_interval, the chosen block window_size, the effective_sample_size, and independence diagnostics (Ljung–Box). Convenience wrappers mean, mean_uncertainty, confidence_interval, and effective_sample_size return just the piece you need.

Ensembles¶

An Ensemble is a collection of DataStream members analysed together. Build one from files or from existing streams:

ens = qnds.Ensemble.from_files(["run01.csv", "run02.csv"], "Q_D/Q_GBD")
result = ens.compute_uncertainty(method="ensemble_average")

Three ensemble estimators are available via method:

Members on different time grids are interpolated to a common grid automatically; empty members are ignored.

Plotting & export¶

The postprocessing layer provides plotting and serialisation. The Plotter returns the Matplotlib objects (a list of (fig, axes)) instead of forcing a display, so plots compose cleanly in notebooks and scripts:

plotter = qnds.Plotter()
plotter.trace_plot(ds, ["Q_D/Q_GBD"])
plotter.steady_state_automatic_plot(ds, ["Q_D/Q_GBD"], batch_size=10)

Exporter displays or converts results (display_dataframe, display_json), and JsonWriter / JsonLoader round-trip a DataStream (including its operation history) to and from JSON.

Workflows¶

The workflow layer packages common multi-step analyses end-to-end:

• RobustWorkflow — robust stationarity assessment and steady-state detection for noisy signals, returning “ball-park” statistics when no clean steady state is found.

• Ensemble workflows — EnsembleAverageWorkflow, EnsembleStatisticsWorkflow, and BatchEnsembleWorkflow orchestrate loading, trimming, averaging, and statistics across many runs.

See the Gallery of examples for runnable, end-to-end demonstrations, and the API Reference for full signatures.