|
libadc-cxx 1.0.0
Structured logging for scientific computing
|
Implementation of builder_api with optional (compile-time) support of MPI. If compiled without MPI, the mpi-related calls devolve to serial behavior. More...
#include <builder.hpp>


Public Member Functions | |
| builder (void *mpi_communicator_p=NULL) | |
| void | add_section (std::string_view name, std::shared_ptr< builder_api > section) |
| copy populated generic section into the builder under specified name. | |
| void | add_header_section (std::string_view application_name) |
| auto-populate the "header" section with application name and required local data | |
| void | add_host_section (int32_t subsections) |
| auto-populate the "host" section based on bitflags. There are many optional subsections covering cpus, gpus, numa, OS. RAM, etc. | |
| void | add_app_data_section (std::shared_ptr< builder_api > app_data) |
| create the "app_data" section with data defined by the application writer. | |
| void | add_model_data_section (std::shared_ptr< builder_api > model_data) |
| populate application run-time physics (re)configuration/result to "model_data" section. For example initial or changes in mesh/particle decomp go here. | |
| void | add_code_section (std::string tag, std::shared_ptr< builder_api > version, std::shared_ptr< builder_api > code_details) |
| void | add_code_configuration_section (std::shared_ptr< builder_api > build_details) |
| Populate build/install "configuration" information such as options enabled. | |
| void | add_exit_data_section (int return_code, std::string status, std::shared_ptr< builder_api > status_details) |
| populate "exit_data" section with code and status stream and user provided details. | |
| void | add_memory_usage_section () |
| populate "memory_usage" section with current host /proc/meminfo data in the style of free(1). | |
| void | add_mpi_section (std::string_view name, void *mpi_comm_p, adc_mpi_field_flags bitflags) |
| add data about a named mpi communicator. In most applications, "mpi_comm_world" is the recommended name. Applications with multiple communicators for data separation can make multiple calls to add_mpi_section with distinct names, such as "comm_ocean" or "comm_atmosphere". | |
| void | add_workflow_section () |
| add data from adc_wfid_ environment variables. The section name is "adc_workflow". | |
| void | add_workflow_children (std::vector< std::string > &child_uuids) |
| add list of child uuids to "adc_workflow" section after add_workflow_section has been called. This call is optional. | |
| void | add_slurm_section () |
| add slurm output environment variable dictionary elements. The section added is named "slurm". | |
| void | add_slurm_section (const std::vector< std::string > &slurmvars) |
| add slurm output environment variable dictionary elements, with the names of additionally desired SLURM output variables. | |
| void | add_gitlab_ci_section () |
| add gitlab_ci environment variable dictionary. The section added is named "gitlab_ci". | |
| std::shared_ptr< builder_api > | get_section (std::string_view name) |
| get the existing named section | |
| std::vector< std::string > | get_section_names () |
| get the names of sections | |
| std::vector< std::string > | get_field_names () |
| get the names of non-section fields in the section | |
| const field | get_value (std::string_view path) |
| get the existing named nested field in the builder. | |
| const char * | get_value_string (std::string_view path) |
| get the existing named nested scalar string value. | |
| int64_t | get_value_int64 (std::string_view path) |
| get the existing named scalar value that can be correctly cast as int64_t | |
| uint64_t | get_value_uint64 (std::string_view path) |
| get the existing named scalar value that can be correctly cast as int64_t | |
| void | add (std::string_view name, bool value) |
| add a named boolean | |
| void | add (std::string_view name, char value) |
| add a named character | |
| void | add (std::string_view name, char16_t value) |
| add a named char16_t | |
| void | add (std::string_view name, char32_t value) |
| add a named char32_t | |
| void | add (std::string_view name, char *value) |
| add null-terminated string | |
| void | add (std::string_view name, const char *value) |
| add null-terminated string | |
| void | add (std::string_view name, std::string &value) |
| add null-terminated string | |
| void | add (std::string_view name, std::string_view value) |
| add null-terminated string | |
| void | add_path (std::string_view name, char *value) |
| add null-terminated string filepath | |
| void | add_path (std::string_view name, const char *value) |
| add null-terminated string filepath | |
| void | add_path (std::string_view name, std::string &value) |
| add null-terminated string filepath | |
| void | add_path (std::string_view name, std::string_view value) |
| add null-terminated string filepath | |
| void | add_json_string (std::string_view name, std::string_view value) |
| add string which is serialized json. | |
| void | add_yaml_string (std::string_view name, std::string_view value) |
| add string which is yaml. | |
| void | add_xml_string (std::string_view name, std::string_view value) |
| add string which is xml. | |
| void | add_number_string (std::string_view name, std::string_view value) |
| add string which is an arbitrary precision decimal number | |
| void | add (std::string_view name, uint8_t value) |
| add named uint8_t | |
| void | add (std::string_view name, uint16_t value) |
| add named uint16_t | |
| void | add (std::string_view name, uint32_t value) |
| add named uint32_t | |
| void | add (std::string_view name, uint64_t value) |
| add named uint64_t | |
| void | add (std::string_view name, int8_t value) |
| add named int8_t | |
| void | add (std::string_view name, int16_t value) |
| add named int16_t | |
| void | add (std::string_view name, int32_t value) |
| add named int32_t | |
| void | add (std::string_view name, int64_t value) |
| add named int64_t | |
| void | add (std::string_view name, float value) |
| add named 32-bit float | |
| void | add (std::string_view name, const std::complex< float > &value) |
| add named 32-bit complex | |
| void | add (std::string_view name, double value) |
| add named 64-bit float | |
| void | add (std::string_view name, const std::complex< double > &value) |
| add named 64-bit complex | |
| void | add (std::string_view name, const struct timeval &tv) |
| add timeval | |
| void | add (std::string_view name, const struct timespec &ts) |
| add timespec | |
| void | add_epoch (std::string_view name, int64_t epoch) |
| add unix epoch seconds (gettimeofday) | |
| void | add_from_pointer_type (std::string_view name, void *, enum scalar_type t) |
| add data from a c pointer | |
| void | add_array (std::string_view name, bool value[], size_t len, std::string_view c) |
| Fixed length arrays of scalar members. | |
| void | add_array (std::string_view name, const char *value, size_t len, std::string_view c) |
| store fixed length array. this is how a character buffer containing nuls is stored. | |
| void | add_array (std::string_view name, char16_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, char32_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, uint8_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, uint16_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, uint32_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, uint64_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, int8_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, int16_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, int32_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, int64_t value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, float value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, double value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, char *value[], size_t len, std::string_view c) |
| Irregular element size array members. | |
| void | add_array (std::string_view name, const char *value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, std::string value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, const std::string value[], size_t len, std::string_view c) |
| void | add_array (std::string_view name, const std::vector< std::string > sv, std::string_view c) |
| void | add_array (std::string_view name, const std::set< std::string > sv, std::string_view container) |
| void | add_array (std::string_view name, const std::list< std::string > sv, std::string_view container) |
| void | add_array_json_string (std::string_view name, const std::string value[], size_t len, std::string_view c) |
| Add Array of strings which are serialized json. | |
| std::string | serialize () |
| convert object to a json string reflecting the section hierarchy. | |
Implementation of builder_api with optional (compile-time) support of MPI. If compiled without MPI, the mpi-related calls devolve to serial behavior.
Definition at line 36 of file builder.hpp.
| adc::builder::builder | ( | void * | mpi_communicator_p = NULL | ) |
Definition at line 423 of file builder.ipp.
|
virtual |
copy populated generic section into the builder under specified name.
| name | key for a new object in the output. |
| section | the value for the object. Note conventional names for certain sections found in other add_*section functions. |
Implements adc::builder_api.
Definition at line 670 of file builder.ipp.
|
virtual |
auto-populate the "header" section with application name and required local data
Implements adc::builder_api.
Definition at line 431 of file builder.ipp.
|
virtual |
auto-populate the "host" section based on bitflags. There are many optional subsections covering cpus, gpus, numa, OS. RAM, etc.
| bitflags | the OR (|) of the desired ADC_HS_*. |
Implements adc::builder_api.
Definition at line 498 of file builder.ipp.
|
virtual |
create the "app_data" section with data defined by the application writer.
Any relationship to previous jobs/higher level workflows goes in app_data, Example: std::shared_ptr< builder_api > app_data = factory.get_builder(); app_data.add("saw_id", getenv("SAW_WORKFLOW_ID"); builder->add_app_data_section(app_data);
Implements adc::builder_api.
Definition at line 570 of file builder.ipp.
|
virtual |
populate application run-time physics (re)configuration/result to "model_data" section. For example initial or changes in mesh/particle decomp go here.
Implements adc::builder_api.
Definition at line 602 of file builder.ipp.
|
virtual |
Auto-populate "code" section with os-derived info at time of call, and user-provided tag string, version subsection, and code_details subsction.
Example: pending.
Implements adc::builder_api.
Definition at line 611 of file builder.ipp.
|
virtual |
Populate build/install "configuration" information such as options enabled.
Example: pending.
Implements adc::builder_api.
Definition at line 636 of file builder.ipp.
|
virtual |
populate "exit_data" section with code and status stream and user provided details.
Example: pending.
Implements adc::builder_api.
Definition at line 648 of file builder.ipp.
|
virtual |
populate "memory_usage" section with current host /proc/meminfo data in the style of free(1).
values included are: mem_total mem_used mem_free mem_shared mem_buffers mem_cache mem_available swap_total swap_used swap_free
Implements adc::builder_api.
Definition at line 577 of file builder.ipp.
|
virtual |
add data about a named mpi communicator. In most applications, "mpi_comm_world" is the recommended name. Applications with multiple communicators for data separation can make multiple calls to add_mpi_section with distinct names, such as "comm_ocean" or "comm_atmosphere".
Implements adc::builder_api.
Definition at line 1338 of file builder.ipp.
|
virtual |
add data from adc_wfid_ environment variables. The section name is "adc_workflow".
The env variables collected are: wfid: ADC_WFID wfid_parent: $ADC_WFID_PARENT wfid_path: $ADC_WFID_PATH
The suggested format of an adc workflow identifier (wfid) is as: uuid -v1 -F STR run at the appropriate scope. For example, when starting numerous processes with mpi under slurm, in the sbatch script before launching anything else do: export ADC_WFID=$(uuid -v1 -F STR) and then make sure it gets propagated to all the processes via the launch mechamism. This ties all the messages from mpi ranks together in adc.
Where a workflow parent (such as an agent launching multiple slurm jobs) can, export ADC_WFID_PARENT=$(uuid -v1 -F STR) and then make sure this value is propagated to the slurm environments.
Where possible (requires coordination at all workflow levels) export ADC_WFID_PATH=(higher_level_wfid_path)/$ADC_WFID the entire task hierarchy identifier can be collected.
Implements adc::builder_api.
Definition at line 1550 of file builder.ipp.
|
virtual |
add list of child uuids to "adc_workflow" section after add_workflow_section has been called. This call is optional.
This call may be repeated if necessary, incrementally building the child list. wfid_children: [ user defined list of ids ]
Where a workflow can track its immediate children, it may substantially improve downstream workflow analyses if the child items can be captured. The result appears in the resulting json as wfid_children: $ADC_WFID_CHILDREN
Implements adc::builder_api.
Definition at line 1597 of file builder.ipp.
|
virtual |
add slurm output environment variable dictionary elements. The section added is named "slurm".
The variables collected from env() are:
Where the values are strings from the corresponding env() values.
Implements adc::builder_api.
Definition at line 1516 of file builder.ipp.
|
virtual |
add slurm output environment variable dictionary elements, with the names of additionally desired SLURM output variables.
Implements adc::builder_api.
Definition at line 1522 of file builder.ipp.
|
virtual |
add gitlab_ci environment variable dictionary. The section added is named "gitlab_ci".
The variables collected from env() are:
Where the values are strings from the corresponding env() values.
Implements adc::builder_api.
Definition at line 1627 of file builder.ipp.
|
virtual |
get the existing named section
Implements adc::builder_api.
Definition at line 685 of file builder.ipp.
|
virtual |
get the names of sections
Implements adc::builder_api.
Definition at line 694 of file builder.ipp.
|
virtual |
get the names of non-section fields in the section
Implements adc::builder_api.
Definition at line 704 of file builder.ipp.
|
virtual |
get the existing named nested field in the builder.
| path | a simple json path such as /a/b/c which resolves to a value added via one of the add* functions. |
Implements adc::builder_api.
Definition at line 1150 of file builder.ipp.
|
virtual |
get the existing named nested scalar string value.
| path | a simple json path such as /a/b/c which resolves to a value added via one of the add* functions. |
Implements adc::builder_api.
Definition at line 1259 of file builder.ipp.
|
virtual |
get the existing named scalar value that can be correctly cast as int64_t
| path | a simple json path such as /a/b/c which resolves to a value added via one of the add* functions. |
Implements adc::builder_api.
Definition at line 1277 of file builder.ipp.
|
virtual |
get the existing named scalar value that can be correctly cast as int64_t
| path | a simple json path such as /a/b/c which resolves to a value added via one of the add* functions. |
Implements adc::builder_api.
Definition at line 1308 of file builder.ipp.
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
add null-terminated string
Implements adc::builder_api.
Definition at line 1693 of file builder.ipp.
|
virtual |
add null-terminated string
Implements adc::builder_api.
Definition at line 1701 of file builder.ipp.
|
virtual |
add null-terminated string
Implements adc::builder_api.
Definition at line 1717 of file builder.ipp.
|
virtual |
add null-terminated string
Implements adc::builder_api.
Definition at line 1709 of file builder.ipp.
|
virtual |
add null-terminated string filepath
Implements adc::builder_api.
Definition at line 1727 of file builder.ipp.
|
virtual |
add null-terminated string filepath
Implements adc::builder_api.
Definition at line 1735 of file builder.ipp.
|
virtual |
add null-terminated string filepath
Implements adc::builder_api.
Definition at line 1751 of file builder.ipp.
|
virtual |
add null-terminated string filepath
Implements adc::builder_api.
Definition at line 1743 of file builder.ipp.
|
virtual |
add string which is serialized json.
Implements adc::builder_api.
Definition at line 1761 of file builder.ipp.
|
virtual |
|
virtual |
|
virtual |
add string which is an arbitrary precision decimal number
Implements adc::builder_api.
Definition at line 1788 of file builder.ipp.
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
add unix epoch seconds (gettimeofday)
Implements adc::builder_api.
Definition at line 1925 of file builder.ipp.
|
virtual |
|
virtual |
Fixed length arrays of scalar members.
Implements adc::builder_api.
Definition at line 2158 of file builder.ipp.
|
virtual |
store fixed length array. this is how a character buffer containing nuls is stored.
Implements adc::builder_api.
Definition at line 2169 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2180 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2191 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2202 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2212 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2222 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2232 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2245 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2255 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2265 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2275 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2285 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2307 of file builder.ipp.
|
virtual |
Irregular element size array members.
Implements adc::builder_api.
Definition at line 2330 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2340 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2350 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
|
virtual |
Implements adc::builder_api.
Definition at line 2370 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2379 of file builder.ipp.
|
virtual |
Implements adc::builder_api.
Definition at line 2388 of file builder.ipp.
|
virtual |
Add Array of strings which are serialized json.
Implements adc::builder_api.
Definition at line 2398 of file builder.ipp.
|
virtual |
convert object to a json string reflecting the section hierarchy.
Implements adc::builder_api.
Definition at line 2422 of file builder.ipp.