|
libadc-cxx 1.0.0
Structured logging for scientific computing
|
Implementation of builder_api with optional (compile-time) support of MPI. If compiled without MPI, the mpi-related calls devolve to serial behavior. More...
#include <builder.hpp>


Public Member Functions | |
| builder (void *mpi_communicator_p=NULL) | |
| void | add_section (std::string_view name, std::shared_ptr< builder_api > section) |
| copy populated generic section into the builder under specified name. | |
| void | add_header_section (std::string_view application_name) |
| auto-populate the "header" section with application name and required local data | |
| void | add_host_section (int32_t subsections) |
| auto-populate the "host" section based on bitflags. There are many optional subsections covering cpus, gpus, numa, OS. RAM, etc. | |
| void | add_app_data_section (std::shared_ptr< builder_api > app_data) |
| create the "app_data" section with data defined by the application writer. | |
| void | add_model_data_section (std::shared_ptr< builder_api > model_data) |
| populate application run-time physics (re)configuration/result to "model_data" section. For example initial or changes in mesh/particle decomp go here. | |
| void | add_code_section (std::string tag, std::shared_ptr< builder_api > version, std::shared_ptr< builder_api > code_details) |
| void | add_code_configuration_section (std::shared_ptr< builder_api > build_details) |
| Populate build/install "configuration" information such as options enabled. | |
| void | add_exit_data_section (int return_code, std::string status, std::shared_ptr< builder_api > status_details) |
| populate "exit_data" section with code and status stream and user provided details. | |
| void | add_memory_usage_section () |
| populate "memory_usage" section with current host /proc/meminfo data in the style of free(1). | |
| void | add_mpi_section (std::string_view name, void *mpi_comm_p, adc_mpi_field_flags bitflags) |
| add data about a named mpi communicator. In most applications, "mpi_comm_world" is the recommended name. Applications with multiple communicators for data separation can make multiple calls to add_mpi_section with distinct names, such as "comm_ocean" or "comm_atmosphere". | |
| void | add_workflow_section () |
| add data from adc_wfid_ environment variables. The section name is "adc_workflow". | |
| void | add_workflow_children (std::vector< std::string > &child_uuids) |
| add list of child uuids to "adc_workflow" section after add_workflow_section has been called. This call is optional. | |
| void | add_slurm_section () |
| add slurm output environment variable dictionary elements. The section added is named "slurm". | |
| void | add_slurm_section (const std::vector< std::string > &slurmvars) |
| add slurm output environment variable dictionary elements, with the names of additionally desired SLURM output variables. | |
| void | add_gitlab_ci_section () |
| add gitlab_ci environment variable dictionary. The section added is named "gitlab_ci". | |
| std::shared_ptr< builder_api > | get_section (std::string_view name) |
| get the existing named section | |
| std::vector< std::string > | get_section_names () |
| get the names of sections | |
| std::vector< std::string > | get_field_names () |
| get the names of non-section fields in the section | |
| const field | get_value (std::string_view name) |
| get the existing named field in the section. | |
| void | add (std::string_view name, bool value) |
| add a named boolean | |
| void | add (std::string_view name, char value) |
| add a named character | |
| void | add (std::string_view name, char16_t value) |
| add a named char16_t | |
| void | add (std::string_view name, char32_t value) |
| add a named char32_t | |
| void | add (std::string_view name, char *value) |
| add null-terminated string | |
| void | add (std::string_view name, const char *value) |
| add null-terminated string | |
| void | add (std::string_view name, std::string &value) |
| add null-terminated string | |
| void | add (std::string_view name, std::string_view value) |
| add null-terminated string | |
| void | add_path (std::string_view name, char *value) |
| add null-terminated string filepath | |
| void | add_path (std::string_view name, const char *value) |
| add null-terminated string filepath | |
| void | add_path (std::string_view name, std::string &value) |
| add null-terminated string filepath | |
| void | add_path (std::string_view name, std::string_view value) |
| add null-terminated string filepath | |
| void | add_json_string (std::string_view name, std::string_view value) |
| add string which is serialized json. | |
| void | add_yaml_string (std::string_view name, std::string_view value) |
| add string which is yaml. | |
| void | add_xml_string (std::string_view name, std::string_view value) |
| add string which is xml. | |
| void | add_number_string (std::string_view name, std::string_view value) |
| add string which is an arbitrary precision decimal number | |
| void | add (std::string_view name, uint8_t value) |
| add named uint8_t | |
| void | add (std::string_view name, uint16_t value) |
| add named uint16_t | |
| void | add (std::string_view name, uint32_t value) |
| add named uint32_t | |
| void | add (std::string_view name, uint64_t value) |
| add named uint64_t | |
| void | add (std::string_view name, int8_t value) |
| add named int8_t | |
| void | add (std::string_view name, int16_t value) |
| add named int16_t | |
| void | add (std::string_view name, int32_t value) |
| add named int32_t | |
| void | add (std::string_view name, int64_t value) |
| add named int64_t | |
| void | add (std::string_view name, float value) |
| add named 32-bit float | |
| void | add (std::string_view name, const std::complex< float > &value) |
| add named 32-bit complex | |
| void | add (std::string_view name, double value) |
| add named 64-bit float | |
| void | add (std::string_view name, const std::complex< double > &value) |
| add named 64-bit complex | |
| void | add (std::string_view name, const struct timeval &tv) |
| add timeval | |
| void | add (std::string_view name, const struct timespec &ts) |
| add timespec | |
| void | add_epoch (std::string_view name, int64_t epoch) |
| add unix epoch seconds (gettimeofday) | |
| void | add_from_pointer_type (std::string_view name, void *, enum scalar_type t) |
| add data from a c pointer | |
| void | add_array (std::string_view name, bool value[], size_t len, std::string_view c) |
| Fixed length arrays of scalar members. | |
| void | add_array (std::string_view name, const char *value, size_t len, std::string_view c) |
| store fixed length array. this is how a character buffer containing nuls is stored. | |
| void | add_array (std::string_view name, char16_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, char32_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, uint8_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, uint16_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, uint32_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, uint64_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, int8_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, int16_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, int32_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, int64_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, float value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, double value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, char *value[], size_t len, std::string_view c) |
| Irregular element size array members. | |
| void | add_array (std::string_view name, const char *value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, std::string value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, const std::string value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, const std::vector< std::string > sv, std::string_view c) |
| void | add_array (std::string_view name, const std::set< std::string > sv, std::string_view container) |
| void | add_array (std::string_view name, const std::list< std::string > sv, std::string_view container) |
| void | add_array_json_string (std::string_view name, const std::string value[], size_t len, std::string_view c) |
| Add Array of strings which are serialized json. | |
| std::string | serialize () |
| convert object to a json string reflecting the section hierarchy. | |
Implementation of builder_api with optional (compile-time) support of MPI. If compiled without MPI, the mpi-related calls devolve to serial behavior.
Definition at line 32 of file builder.hpp.
| adc::builder::builder | ( | void * | mpi_communicator_p = NULL | ) |
Definition at line 383 of file builder.ipp.
|
virtual |
copy populated generic section into the builder under specified name.
| name | key for a new object in the output. |
| section | the value for the object. Note conventional names for certain sections found in other add_*section functions. |
Implements adc::builder_api.
Definition at line 599 of file builder.ipp.
|
virtual |
auto-populate the "header" section with application name and required local data
Implements adc::builder_api.
Definition at line 387 of file builder.ipp.
|
virtual |
auto-populate the "host" section based on bitflags. There are many optional subsections covering cpus, gpus, numa, OS. RAM, etc.
| bitflags | the OR (|) of the desired ADC_HS_*. |
Implements adc::builder_api.
Definition at line 453 of file builder.ipp.
|
virtual |
create the "app_data" section with data defined by the application writer.
Any relationship to previous jobs/higher level workflows goes in app_data, Example: std::shared_ptr< builder_api > app_data = factory.get_builder(); app_data.add("saw_id", getenv("SAW_WORKFLOW_ID"); builder->add_app_data_section(app_data);
Implements adc::builder_api.
Definition at line 518 of file builder.ipp.
|
virtual |
populate application run-time physics (re)configuration/result to "model_data" section. For example initial or changes in mesh/particle decomp go here.
Implements adc::builder_api.
Definition at line 549 of file builder.ipp.
|
virtual |
Auto-populate "code" section with os-derived info at time of call, and user-provided tag string, version subsection, and code_details subsction.
Example: pending.
Implements adc::builder_api.
Definition at line 557 of file builder.ipp.
|
virtual |
Populate build/install "configuration" information such as options enabled.
Example: pending.
Implements adc::builder_api.
Definition at line 580 of file builder.ipp.
|
virtual |
populate "exit_data" section with code and status stream and user provided details.
Example: pending.
Implements adc::builder_api.
Definition at line 587 of file builder.ipp.
|
virtual |
populate "memory_usage" section with current host /proc/meminfo data in the style of free(1).
values included are: mem_total mem_used mem_free mem_shared mem_buffers mem_cache mem_available swap_total swap_used swap_free
Implements adc::builder_api.
Definition at line 524 of file builder.ipp.
|
virtual |
add data about a named mpi communicator. In most applications, "mpi_comm_world" is the recommended name. Applications with multiple communicators for data separation can make multiple calls to add_mpi_section with distinct names, such as "comm_ocean" or "comm_atmosphere".
Implements adc::builder_api.
Definition at line 1163 of file builder.ipp.
|
virtual |
add data from adc_wfid_ environment variables. The section name is "adc_workflow".
The env variables collected are: wfid: ADC_WFID wfid_parent: $ADC_WFID_PARENT wfid_path: $ADC_WFID_PATH
The suggested format of an adc workflow identifier (wfid) is as: uuid -v1 -F STR run at the appropriate scope. For example, when starting numerous processes with mpi under slurm, in the sbatch script before launching anything else do: export ADC_WFID=$(uuid -v1 -F STR) and then make sure it gets propagated to all the processes via the launch mechamism. This ties all the messages from mpi ranks together in adc.
Where a workflow parent (such as an agent launching multiple slurm jobs) can, export ADC_WFID_PARENT=$(uuid -v1 -F STR) and then make sure this value is propagated to the slurm environments.
Where possible (requires coordination at all workflow levels) export ADC_WFID_PATH=(higher_level_wfid_path)/$ADC_WFID the entire task hierarchy identifier can be collected.
Implements adc::builder_api.
Definition at line 1375 of file builder.ipp.
|
virtual |
add list of child uuids to "adc_workflow" section after add_workflow_section has been called. This call is optional.
This call may be repeated if necessary, incrementally building the child list. wfid_children: [ user defined list of ids ]
Where a workflow can track its immediate children, it may substantially improve downstream workflow analyses if the child items can be captured. The result appears in the resulting json as wfid_children: $ADC_WFID_CHILDREN
Implements adc::builder_api.
Definition at line 1422 of file builder.ipp.
|
virtual |
add slurm output environment variable dictionary elements. The section added is named "slurm".
The variables collected from env() are:
Where the values are strings from the corresponding env() values.
Implements adc::builder_api.
Definition at line 1341 of file builder.ipp.
|
virtual |
add slurm output environment variable dictionary elements, with the names of additionally desired SLURM output variables.
Implements adc::builder_api.
Definition at line 1347 of file builder.ipp.
|
virtual |
add gitlab_ci environment variable dictionary. The section added is named "gitlab_ci".
The variables collected from env() are:
Where the values are strings from the corresponding env() values.
Implements adc::builder_api.
Definition at line 1452 of file builder.ipp.
|
virtual |
get the existing named section
Implements adc::builder_api.
Definition at line 609 of file builder.ipp.
|
virtual |
get the names of sections
Implements adc::builder_api.
Definition at line 618 of file builder.ipp.
|
virtual |
get the names of non-section fields in the section
Implements adc::builder_api.
Definition at line 627 of file builder.ipp.
|
virtual |
get the existing named field in the section.
Implements adc::builder_api.
Definition at line 1073 of file builder.ipp.
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
add null-terminated string
Implements adc::builder_api.
Definition at line 1514 of file builder.ipp.
|
virtual |
add null-terminated string
Implements adc::builder_api.
Definition at line 1521 of file builder.ipp.
|
virtual |
add null-terminated string
Implements adc::builder_api.
Definition at line 1535 of file builder.ipp.
|
virtual |
add null-terminated string
Implements adc::builder_api.
Definition at line 1528 of file builder.ipp.
|
virtual |
add null-terminated string filepath
Implements adc::builder_api.
Definition at line 1544 of file builder.ipp.
|
virtual |
add null-terminated string filepath
Implements adc::builder_api.
Definition at line 1551 of file builder.ipp.
|
virtual |
add null-terminated string filepath
Implements adc::builder_api.
Definition at line 1565 of file builder.ipp.
|
virtual |
add null-terminated string filepath
Implements adc::builder_api.
Definition at line 1558 of file builder.ipp.
|
virtual |
add string which is serialized json.
Implements adc::builder_api.
Definition at line 1574 of file builder.ipp.
|
virtual |
|
virtual |
|
virtual |
add string which is an arbitrary precision decimal number
Implements adc::builder_api.
Definition at line 1598 of file builder.ipp.
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
add unix epoch seconds (gettimeofday)
Implements adc::builder_api.
Definition at line 1719 of file builder.ipp.
|
virtual |
|
virtual |
Fixed length arrays of scalar members.
Implements adc::builder_api.
Definition at line 1950 of file builder.ipp.
|
virtual |
store fixed length array. this is how a character buffer containing nuls is stored.
Implements adc::builder_api.
Definition at line 1960 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 1970 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 1980 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 1990 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 1999 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2008 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2017 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2029 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2038 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2047 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2056 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2065 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2085 of file builder.ipp.
|
virtual |
Irregular element size array members.
Implements adc::builder_api.
Definition at line 2106 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2115 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2124 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
|
virtual |
Implements adc::builder_api.
Definition at line 2142 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2150 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2158 of file builder.ipp.
|
virtual |
Add Array of strings which are serialized json.
Implements adc::builder_api.
Definition at line 2167 of file builder.ipp.
|
virtual |
convert object to a json string reflecting the section hierarchy.
Implements adc::builder_api.
Definition at line 2190 of file builder.ipp.