8.1. Data model¶
Documentation on PEAT’s internal model for structuring and managing data from devices (a.k.a “device data”).
8.1.1. Working with data¶
There are two ways to store and retrieve data:
Directly via class attributes:
dev.os.version = "7"Using
DeviceData.store()with a model class instance:dev.store("interface", Interface(ip="10.10.10.10"))
Simple attribute values such as architecture or type should be assigned directly, e.g. dev.architecture = "x86_64".
Complex attributes that contain objects, such as interfaces (which is a list of Interface), should be set using DeviceData.store().
General data can be retrieved directly via attribute access, e.g. os_ver = dev.os.version. Complex objects (such as “services”) are easily accessed using the DeviceData.retrieve() helper method, which will search and filter objects based on the desired attributes, e.g. the IP address or port of a interface. However, they can also be accessed directly as regular lists, if desired.
8.1.2. DeviceData¶
- DeviceData[source]¶
Container and manager of all data about a device, e.g. name, IP address, version, etc.
Note
If unset, the
idattribute on this object will be set to the IP of the first Interface added viastore().There are two main methods of storing and retrieving data:
Directly via class attributes. This should be used for most operations.
Via
store()andretrieve(). These are used for containers of objects, e.g.dev.interfaceordev.event".
Note
See the documentation for
store()andretrieve()for detailed examples of how to use those methods.Storing data:
Direct assignment:
dev.os.version = "7"Storing to a list:
dev.store("interface", Interface(ip="192.0.2.10"))
Reading data:
General data:
value = dev.os.versionList of data:
value = dev.retrieve("interface", {"ip": "192.0.2.10"})
Methods for exporting data:
dict()json()
Note
The device can be module or component of a larger system, e.g. a module in a PLC or a wireless add-on module on a power meter. The
modulefield is an example of this use case (alistofDeviceData).Show JSON schema
{ "$ref": "#/definitions/DeviceData", "definitions": { "Hash": { "title": "Hash", "description": "Hashes of raw data or a file.\n\n.. note::\n All hashes are uppercase hexadecimal strings, per :term:`ECS`", "type": "object", "properties": { "md5": { "title": "MD5 hash", "minLength": 32, "maxLength": 32, "type": "string" }, "sha1": { "title": "SHA1 hash", "minLength": 40, "maxLength": 40, "type": "string" }, "sha256": { "title": "SHA256 hash", "minLength": 64, "maxLength": 64, "type": "string" }, "sha512": { "title": "SHA512 hash", "minLength": 128, "maxLength": 128, "type": "string" } }, "additionalProperties": false }, "File": { "title": "File", "description": "Contextual information and metadata for a file.\n\nThe file could be on disk, in memory, a directory, or simply\nrepresent an artifact that's known to be on the device but\nPEAT doesn't have the ability to access.", "type": "object", "properties": { "created": { "title": "Created", "type": "string", "format": "date-time" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "device": { "title": "Device", "default": "", "type": "string" }, "directory": { "title": "Directory", "default": "", "type": "string" }, "extension": { "title": "Extension", "default": "", "type": "string" }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "local_path": { "title": "Local Path", "type": "string", "format": "path" }, "path": { "title": "Path", "type": "string", "format": "path" }, "peat_module": { "title": "Peat Module", "default": "", "type": "string" }, "gid": { "title": "Gid", "default": "", "type": "string" }, "group": { "title": "Group", "default": "", "type": "string" }, "mime_type": { "title": "Mime Type", "default": "", "type": "string" }, "mode": { "title": "Mode", "default": "", "type": "string" }, "mtime": { "title": "Mtime", "type": "string", "format": "date-time" }, "name": { "title": "Name", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "type": "string", "format": "binary" }, "owner": { "title": "Owner", "default": "", "type": "string" }, "size": { "title": "Size", "minimum": 0, "type": "integer" }, "target_path": { "title": "Target Path", "type": "string", "format": "path" }, "type": { "title": "Type", "default": "", "enum": [ "file", "dir", "symlink", "" ], "type": "string" }, "uid": { "title": "Uid", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false }, "Firmware": { "title": "Firmware", "description": "Device firmware.", "type": "object", "properties": { "checksum": { "title": "Checksum", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "file": { "title": "File", "default": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "allOf": [ { "$ref": "#/definitions/File" } ] }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "id": { "title": "Id", "default": "", "type": "string" }, "last_updated": { "title": "Last Updated", "type": "string", "format": "date-time" }, "original": { "title": "Original", "default": "", "type": "string", "format": "binary" }, "revision": { "title": "Revision", "default": "", "type": "string" }, "release_date": { "title": "Release Date", "type": "string", "format": "date-time" }, "timestamp": { "title": "Timestamp", "type": "string", "format": "date-time" }, "version": { "title": "Version", "default": "", "type": "string" } }, "additionalProperties": false }, "Vendor": { "title": "Vendor", "description": "Identifies a device vendor (SEL, Schneider Electric, Siemens, etc).", "type": "object", "properties": { "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" } }, "additionalProperties": false }, "Description": { "title": "Description", "description": "Identifying information such as vendor, brand, and model.", "type": "object", "properties": { "brand": { "title": "Brand", "default": "", "type": "string" }, "contact_info": { "title": "Contact Info", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "full": { "title": "Full", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "model": { "title": "Model", "default": "", "type": "string" }, "product": { "title": "Product", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "vendor": { "title": "Vendor", "default": { "id": "", "name": "" }, "allOf": [ { "$ref": "#/definitions/Vendor" } ] } }, "additionalProperties": false }, "Hardware": { "title": "Hardware", "description": "Hardware information of the device, e.g. amount of :term:`RAM`.", "type": "object", "properties": { "cpu": { "title": "Cpu", "default": { "brand": "", "contact_info": "", "description": "", "full": "", "model": "", "product": "", "vendor": { "id": "", "name": "" } }, "allOf": [ { "$ref": "#/definitions/Description" } ] }, "id": { "title": "Id", "default": "", "type": "string" }, "storage_available": { "title": "Storage Available", "minimum": 0, "type": "integer" }, "storage_usage": { "title": "Storage Usage", "minimum": 0, "type": "integer" }, "storage_total": { "title": "Storage Total", "minimum": 0, "type": "integer" }, "storage_type": { "title": "Storage Type", "default": "", "type": "string" }, "memory_available": { "title": "Memory Available", "minimum": 0, "type": "integer" }, "memory_usage": { "title": "Memory Usage", "minimum": 0, "type": "integer" }, "memory_total": { "title": "Memory Total", "minimum": 0, "type": "integer" }, "memory_type": { "title": "Memory Type", "default": "", "type": "string" }, "revision": { "title": "Revision", "default": "", "type": "string" }, "version": { "title": "Version", "default": "", "type": "string" } }, "additionalProperties": false }, "OS": { "title": "OS", "description": "Operating System (OS) information, such as the name and version.", "type": "object", "properties": { "family": { "title": "Family", "default": "", "type": "string" }, "full": { "title": "Full", "default": "", "type": "string" }, "kernel": { "title": "Kernel", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "timestamp": { "title": "Timestamp", "type": "string", "format": "date-time" }, "vendor": { "title": "Vendor", "default": { "id": "", "name": "" }, "allOf": [ { "$ref": "#/definitions/Vendor" } ] }, "version": { "title": "Version", "default": "", "type": "string" } }, "additionalProperties": false }, "LatLon": { "title": "LatLon", "description": "Latitude and Longitude (geographical coordinates).", "type": "object", "properties": { "lat": { "title": "Latitude", "elastic_type": "double", "minimum": -90.0, "maximum": 90.0, "type": "number" }, "lon": { "title": "Longitude", "elastic_type": "double", "minimum": -180.0, "maximum": 180.0, "type": "number" } }, "additionalProperties": false }, "Geo": { "title": "Geo", "description": "Geolocation information (the device's physical location).", "type": "object", "properties": { "city_name": { "title": "City Name", "default": "", "type": "string" }, "country_name": { "title": "Country Name", "default": "", "type": "string" }, "location": { "title": "Location", "default": { "lat": null, "lon": null }, "elastic_type": "geo_point", "allOf": [ { "$ref": "#/definitions/LatLon" } ] }, "name": { "title": "Name", "default": "", "type": "string" }, "timezone": { "title": "Timezone", "default": "", "type": "string" } }, "additionalProperties": false }, "Logic": { "title": "Logic", "description": "What the device has been programmed to do (it's \"logic\").\n\nIn a :term:`PLC`, the logic is one or more of the five `IEC 61181-3\n<https://en.wikipedia.org/wiki/IEC_61131-3>`_ languages:\n\n- Ladder Diagram (LD)\n- Function Block Diagram (FBD)\n- Structured Text (ST)\n- Instruction List (IL)\n- Sequential Function Chart (SFC)\n\nIn a Relay, the logic is the protection schemes.\n\nIn a Power Meter, the logic is the programmed metering/monitoring setpoints.\n\n.. note::\n Logic is separate from protocol register mappings or values, such as\n Modbus or DNP3, as well as memory values. There is sometimes overlap,\n as some devices have been known to store their logic as e.g. a set\n of Modbus registers.", "type": "object", "properties": { "author": { "title": "Author", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "created": { "title": "Created", "type": "string", "format": "date-time" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "file": { "title": "File", "default": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "allOf": [ { "$ref": "#/definitions/File" } ] }, "formats": { "title": "Formats", "default": {}, "elastic_type": "nested", "type": "object", "additionalProperties": { "anyOf": [ { "anyOf": [ { "type": "string", "format": "binary" }, { "type": "string" } ] }, { "type": "object" } ] } }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "id": { "title": "Id", "default": "", "type": "string" }, "last_updated": { "title": "Last Updated", "type": "string", "format": "date-time" }, "name": { "title": "Name", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "original": { "title": "Original", "default": "", "elastic_type": "text", "type": "string" }, "parsed": { "title": "Parsed", "default": "", "elastic_type": "text", "type": "string" } }, "additionalProperties": false }, "Service": { "title": "Service", "description": "Communication protocol \"service\" configured or running on the device.\n\nServices can be over a variety of transports, including IP/Ethernet,\nserial direct, cellular, serial bus, field bus, etc.", "type": "object", "properties": { "configured_port": { "title": "Configured Port", "minimum": 0, "maximum": 65535, "type": "integer" }, "enabled": { "title": "Enabled", "type": "boolean" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "listen_address": { "title": "Listen Address", "default": "", "elastic_type": "ip", "type": "string" }, "listen_interface": { "title": "Listen Interface", "default": "", "type": "string" }, "process_name": { "title": "Process Name", "default": "", "type": "string" }, "process_pid": { "title": "Process Pid", "minimum": 0, "type": "integer" }, "port": { "title": "Port", "minimum": 1, "maximum": 65535, "type": "integer" }, "protocol": { "title": "Protocol", "default": "", "type": "string" }, "protocol_id": { "title": "Protocol Id", "default": "", "type": "string" }, "role": { "title": "Role", "default": "", "type": "string" }, "status": { "title": "Status", "default": "", "enum": [ "open", "closed", "verified", "" ], "type": "string" }, "transport": { "title": "Transport", "default": "", "type": "string" } }, "additionalProperties": false }, "Interface": { "title": "Interface", "description": "Communication interface, such as a Ethernet port or Serial link.\n\n.. note::\n Currently, the ``ip``, ``subnet_mask``, and ``gateway`` fields are\n assumed to be :term:`IP` version 4 (IPv4). However, they can and\n will hold IPv6 values in the future when PEAT adds IPv6 support.", "type": "object", "properties": { "alias": { "title": "Alias", "default": "", "type": "string" }, "application": { "title": "Application", "default": "", "type": "string" }, "connected": { "title": "Connected", "type": "boolean" }, "description": { "title": "Description", "default": { "brand": "", "contact_info": "", "description": "", "full": "", "model": "", "product": "", "vendor": { "id": "", "name": "" } }, "allOf": [ { "$ref": "#/definitions/Description" } ] }, "duplex": { "title": "Duplex", "default": "", "enum": [ "half", "full", "auto", "" ], "type": "string" }, "enabled": { "title": "Enabled", "type": "boolean" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "name": { "title": "Name", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "type": "string" }, "hostname": { "title": "Hostname", "default": "", "type": "string" }, "mac": { "title": "Mac", "default": "", "type": "string" }, "mac_vendor": { "title": "Mac Vendor", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "mtu": { "title": "Mtu", "elastic_type": "integer", "exclusiveMinimum": 0, "type": "integer" }, "physical": { "title": "Physical", "type": "boolean" }, "promiscuous_mode": { "title": "Promiscuous Mode", "type": "boolean" }, "speed": { "title": "Speed", "elastic_type": "integer", "minimum": 0, "type": "integer" }, "uptime": { "title": "Uptime", "type": "number", "format": "time-delta" }, "hardware_mac": { "title": "Hardware Mac", "default": "", "type": "string" }, "id": { "title": "Id", "default": "", "type": "string" }, "ip": { "title": "Ip", "default": "", "elastic_type": "ip", "type": "string" }, "subnet_mask": { "title": "Subnet Mask", "default": "", "elastic_type": "ip", "type": "string" }, "gateway": { "title": "Gateway", "default": "", "elastic_type": "ip", "type": "string" }, "serial_port": { "title": "Serial Port", "default": "", "type": "string" }, "baudrate": { "title": "Baudrate", "elastic_type": "integer", "exclusiveMinimum": 0, "type": "integer" }, "data_bits": { "title": "Data Bits", "elastic_type": "byte", "minimum": 0, "type": "integer" }, "parity": { "title": "Parity", "default": "", "enum": [ "none", "even", "odd", "" ], "type": "string" }, "stop_bits": { "title": "Stop Bits", "elastic_type": "byte", "minimum": 0, "type": "integer" }, "flow_control": { "title": "Flow Control", "default": "", "type": "string" }, "services": { "title": "Services", "default": [], "type": "array", "items": { "$ref": "#/definitions/Service" } }, "version": { "title": "Version", "default": "", "type": "string" } }, "additionalProperties": false }, "SSHKey": { "title": "SSHKey", "description": "SSH keys (public or private).", "type": "object", "properties": { "description": { "title": "Description", "default": "", "type": "string" }, "file": { "title": "File", "default": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "allOf": [ { "$ref": "#/definitions/File" } ] }, "host": { "title": "Host", "default": "", "type": "string" }, "id": { "title": "Id", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "enum": [ "public", "" ], "type": "string" }, "user": { "title": "User", "default": "", "type": "string" } }, "additionalProperties": false }, "Related": { "title": "Related", "description": "Information that is related to a device or interface, or was found on the device.", "type": "object", "properties": { "emails": { "title": "Emails", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "files": { "title": "Files", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "hash": { "title": "Hash", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "hosts": { "title": "Hosts", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "ip": { "title": "Ip", "default": [], "elastic_type": "ip", "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "mac": { "title": "Mac", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "ports": { "title": "Ports", "default": [], "type": "array", "items": { "type": "integer", "minimum": 1, "maximum": 65535 }, "uniqueItems": true }, "protocols": { "title": "Protocols", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "process": { "title": "Process", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "roles": { "title": "Roles", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "urls": { "title": "Urls", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "user": { "title": "User", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true } }, "additionalProperties": false }, "Register": { "title": "Register", "description": "Configured I/O protocol data point (\"registers\"), e.g. DNP3 or Modbus/TCP.", "type": "object", "properties": { "address": { "title": "Address", "default": "", "type": "string" }, "data_type": { "title": "Data Type", "default": "", "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": "text", "type": "string" }, "enabled": { "title": "Enabled", "type": "boolean" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "group": { "title": "Group", "default": "", "type": "string" }, "io": { "title": "Io", "default": "", "type": "string" }, "measurement_type": { "title": "Measurement Type", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "protocol": { "title": "Protocol", "default": "", "type": "string" }, "read_write": { "title": "Read Write", "default": "", "enum": [ "read", "write", "read_write", "" ], "type": "string" }, "tag": { "title": "Tag", "default": "", "type": "string" } }, "additionalProperties": false }, "Tag": { "title": "Tag", "description": "Variable in a :term:`ICS`/:term:`OT` device, often mapping to physical I/O.\n\nThese are commonly stored in a \"tag database\" in a :term:`SCADA` system or\nthe configuration of a device.", "type": "object", "properties": { "address": { "title": "Address", "default": "", "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": "text", "type": "string" }, "io": { "title": "Io", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "type": "string" } }, "additionalProperties": false }, "IO": { "title": "IO", "description": "Physical Input/Output (I/O) connections on a device.\n\nPhysical I/O points are distinct from :class:`~peat.data.models.Register`,\nwhich handle communication protocols and may not necessarily map to physical I/O.\n\nPhysical I/O points are typically referenced by a :class:`~peat.data.models.Tag`,\nthough this may not always be the case.\n\nOn module-based devices like a :term:`PLC`, Physical I/O points may be associated\nwith a module, however this may not always be the case.", "type": "object", "properties": { "address": { "title": "Address", "default": "", "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": "text", "type": "string" }, "direction": { "title": "Direction", "default": "", "enum": [ "input", "output", "" ], "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "type": "string" }, "slot": { "title": "Slot", "default": [], "type": "array", "items": { "type": "string" } } }, "additionalProperties": false }, "Event": { "title": "Event", "description": "Device log entry, such as logins, metering reads, or system events.", "type": "object", "properties": { "action": { "title": "Action", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "category": { "title": "Category", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "created": { "title": "Created", "type": "string", "format": "date-time" }, "dataset": { "title": "Dataset", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "id": { "title": "Id", "default": "", "type": "string" }, "ingested": { "title": "Ingested", "type": "string", "format": "date-time" }, "kind": { "title": "Kind", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "message": { "title": "Message", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "module": { "title": "Module", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "elastic_type": "text", "type": "string" }, "outcome": { "title": "Outcome", "default": "", "type": "string" }, "provider": { "title": "Provider", "default": "", "type": "string" }, "sequence": { "title": "Sequence", "type": "integer" }, "severity": { "title": "Severity", "default": "", "type": "string" }, "timezone": { "title": "Timezone", "default": "", "type": "string" }, "type": { "title": "Type", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true } }, "additionalProperties": false }, "Memory": { "title": "Memory", "description": "Physical memory values (e.g. :term:`RAM`, EEPROM).", "type": "object", "properties": { "address": { "title": "Address", "default": "", "type": "string" }, "created": { "title": "Created", "type": "string", "format": "date-time" }, "dataset": { "title": "Dataset", "default": "", "type": "string" }, "device": { "title": "Device", "default": "", "type": "string" }, "process": { "title": "Process", "default": "", "type": "string" }, "size": { "title": "Size", "type": "integer" }, "value": { "title": "Value", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false }, "User": { "title": "User", "description": "Information describing a user on a device.", "type": "object", "properties": { "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "domain": { "title": "Domain", "default": "", "type": "string" }, "email": { "title": "Email", "default": "", "type": "string" }, "full_name": { "title": "Full Name", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "permissions": { "title": "Permissions", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "roles": { "title": "Roles", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "uid": { "title": "Uid", "default": "", "type": "string" }, "gid": { "title": "Gid", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false }, "CertEntity": { "title": "CertEntity", "description": "Issuer or Subject in a x509 certificate.\n\n`ECS documentation: x509 Certificate Fields <https://www.elastic.co/guide/en/ecs/current/ecs-x509.html>`__", "type": "object", "properties": { "common_name": { "title": "Common Name", "default": "", "type": "string" }, "country": { "title": "Country", "default": "", "type": "string" }, "distinguished_name": { "title": "Distinguished Name", "default": "", "type": "string" }, "locality": { "title": "Locality", "default": "", "type": "string" }, "organization": { "title": "Organization", "default": "", "type": "string" }, "organizational_unit": { "title": "Organizational Unit", "default": "", "type": "string" }, "state_or_province": { "title": "State Or Province", "default": "", "type": "string" } }, "additionalProperties": false }, "X509": { "title": "X509", "description": "x509 certificate.\n\n`ECS documentation: x509 Certificate Fields <https://www.elastic.co/guide/en/ecs/current/ecs-x509.html>`__", "type": "object", "properties": { "alternative_names": { "title": "Alternative Names", "default": [], "type": "array", "items": { "type": "string" } }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "issuer": { "title": "Issuer", "default": { "common_name": "", "country": "", "distinguished_name": "", "locality": "", "organization": "", "organizational_unit": "", "state_or_province": "" }, "allOf": [ { "$ref": "#/definitions/CertEntity" } ] }, "not_after": { "title": "Not After", "type": "string", "format": "date-time" }, "not_before": { "title": "Not Before", "type": "string", "format": "date-time" }, "original": { "title": "Original", "default": "", "type": "string" }, "public_key_algorithm": { "title": "Public Key Algorithm", "default": "", "type": "string" }, "public_key_curve": { "title": "Public Key Curve", "default": "", "type": "string" }, "public_key_exponent": { "title": "Public Key Exponent", "minimum": 0, "type": "integer" }, "public_key_size": { "title": "Public Key Size", "minimum": 0, "type": "integer" }, "serial_number": { "title": "Serial Number", "default": "", "type": "string" }, "signature_algorithm": { "title": "Signature Algorithm", "default": "", "type": "string" }, "subject": { "title": "Subject", "default": { "common_name": "", "country": "", "distinguished_name": "", "locality": "", "organization": "", "organizational_unit": "", "state_or_province": "" }, "allOf": [ { "$ref": "#/definitions/CertEntity" } ] }, "version_number": { "title": "Version Number", "default": "", "type": "string" } }, "additionalProperties": false }, "UEFIFile": { "title": "UEFIFile", "description": "UEFI model that specifically labels objects from a UEFIExtract report file.\nThis model is different because it includes only the SPI file system\nincluded in an SPI file dump\ntype is the type of entry. Examples are \"Region, Volume\"\nsubtype is the subtype of the entry. Can be blank, can be empty or invalid\nbase is the start of location in memory in HEX\nSize is the end of location in memory in HEX\nCRC32 is the calculate crc32 for the file\nName is the name of the file\npath is the path of the file since the dumps are given in a file like\nstructure", "type": "object", "properties": { "type": { "title": "Type", "default": "", "type": "string" }, "subtype": { "title": "Subtype", "default": "", "type": "string" }, "base": { "title": "Base", "default": "", "type": "string" }, "size": { "title": "Size", "default": "", "type": "string" }, "crc32": { "title": "Crc32", "default": "", "type": "string" }, "guid": { "title": "Guid", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "path": { "title": "Path", "default": "", "type": "string" }, "created": { "title": "Created", "type": "string", "format": "date-time" } }, "additionalProperties": false }, "UEFIHash": { "title": "UEFIHash", "description": "UEFI model that specifically labels objects from a UEFI file hash file.\nThis model is different because it includes all file systems, not just\nthe EFI File system\n\nFile system is either FS0 or FS1\npathname is the pathname of the files in the file system\nhash is the SHA256 hash of the files computed via python script", "type": "object", "properties": { "file_system": { "title": "File System", "default": "", "type": "string" }, "pathname": { "title": "Pathname", "default": "", "type": "string" }, "hash": { "title": "Hash", "default": "", "type": "string" } }, "additionalProperties": false }, "DeviceData": { "title": "DeviceData", "description": "Container and manager of all data about a device,\ne.g. name, :term:`IP` address, version, etc.\n\n.. note::\n If unset, the :attr:`~peat.data.models.DeviceData.id` attribute on this\n object will be set to the IP of the first Interface added via\n :meth:`~peat.data.models.DeviceData.store`.\n\nThere are two main methods of storing and retrieving data:\n\n- Directly via class attributes. This should be used for most operations.\n- Via :meth:`~peat.data.models.DeviceData.store` and\n :meth:`~peat.data.models.DeviceData.retrieve`.\n These are used for containers of objects,\n e.g. ``dev.interface`` or ``dev.event\"``.\n\n.. note::\n See the documentation for :meth:`~peat.data.models.DeviceData.store`\n and :meth:`~peat.data.models.DeviceData.retrieve`\n for detailed examples of how to use those methods.\n\nStoring data:\n\n- Direct assignment: ``dev.os.version = \"7\"``\n- Storing to a list: ``dev.store(\"interface\", Interface(ip=\"192.0.2.10\"))``\n\nReading data:\n\n- General data: ``value = dev.os.version``\n- List of data: ``value = dev.retrieve(\"interface\", {\"ip\": \"192.0.2.10\"})``\n\nMethods for exporting data:\n\n- :meth:`~peat.data.models.DeviceData.export`\n- :meth:`~peat.data.models.DeviceData.export_summary`\n- :meth:`~peat.data.models.DeviceData.elastic`\n- :meth:`~peat.data.models.DeviceData.dict`\n- :meth:`~peat.data.models.DeviceData.json`\n- :meth:`~peat.data.models.DeviceData.export_to_elastic`\n- :meth:`~peat.data.models.DeviceData.export_to_files`\n\n.. note::\n The device can be module or component of a larger system, e.g. a module\n in a :term:`PLC` or a wireless add-on module on a power meter. The\n :attr:`~peat.data.models.DeviceData.module` field is an example\n of this use case (a :class:`list` of :class:`~peat.data.models.DeviceData`).", "type": "object", "properties": { "successful_pulls": { "title": "Successful Pulls", "default": {}, "type": "object" }, "architecture": { "title": "Architecture", "default": "", "type": "string" }, "boot_firmware": { "title": "Boot Firmware", "default": { "checksum": "", "extra": {}, "file": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "id": "", "last_updated": null, "original": "", "revision": "", "release_date": null, "timestamp": null, "version": "" }, "allOf": [ { "$ref": "#/definitions/Firmware" } ] }, "description": { "title": "Description", "default": { "brand": "", "contact_info": "", "description": "", "full": "", "model": "", "product": "", "vendor": { "id": "", "name": "" } }, "allOf": [ { "$ref": "#/definitions/Description" } ] }, "endian": { "title": "Endian", "default": "", "enum": [ "big", "little", "" ], "type": "string" }, "firmware": { "title": "Firmware", "default": { "checksum": "", "extra": {}, "file": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "id": "", "last_updated": null, "original": "", "revision": "", "release_date": null, "timestamp": null, "version": "" }, "allOf": [ { "$ref": "#/definitions/Firmware" } ] }, "hardware": { "title": "Hardware", "default": { "cpu": { "brand": "", "contact_info": "", "description": "", "full": "", "model": "", "product": "", "vendor": { "id": "", "name": "" } }, "id": "", "storage_available": null, "storage_usage": null, "storage_total": null, "storage_type": "", "memory_available": null, "memory_usage": null, "memory_total": null, "memory_type": "", "revision": "", "version": "" }, "allOf": [ { "$ref": "#/definitions/Hardware" } ] }, "hostname": { "title": "Hostname", "default": "", "type": "string" }, "id": { "title": "Id", "default": "", "type": "string" }, "ip": { "title": "Ip", "default": "", "elastic_type": "ip", "type": "string" }, "mac": { "title": "Mac", "default": "", "type": "string" }, "mac_vendor": { "title": "Mac Vendor", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "serial_port": { "title": "Serial Port", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "label": { "title": "Label", "default": "", "type": "string" }, "comment": { "title": "Comment", "default": "", "type": "string" }, "part_number": { "title": "Part Number", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "type": "string" }, "serial_number": { "title": "Serial Number", "default": "", "type": "string" }, "manufacturing_date": { "title": "Manufacturing Date", "type": "string", "format": "date-time" }, "run_mode": { "title": "Run Mode", "default": "", "type": "string" }, "slot": { "title": "Slot", "default": "", "type": "string" }, "start_time": { "title": "Start Time", "type": "string", "format": "date-time" }, "status": { "title": "Status", "default": "", "type": "string" }, "uptime": { "title": "Uptime", "type": "number", "format": "time-delta" }, "os": { "title": "Os", "default": { "family": "", "full": "", "kernel": "", "name": "", "timestamp": null, "vendor": { "id": "", "name": "" }, "version": "" }, "allOf": [ { "$ref": "#/definitions/OS" } ] }, "geo": { "title": "Geo", "default": { "city_name": "", "country_name": "", "location": { "lat": null, "lon": null }, "name": "", "timezone": "" }, "allOf": [ { "$ref": "#/definitions/Geo" } ] }, "logic": { "title": "Logic", "default": { "author": "", "created": null, "description": "", "file": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "formats": {}, "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "id": "", "last_updated": null, "name": "", "original": "", "parsed": "" }, "allOf": [ { "$ref": "#/definitions/Logic" } ] }, "files": { "title": "Files", "default": [], "type": "array", "items": { "$ref": "#/definitions/File" } }, "interface": { "title": "Interface", "default": [], "type": "array", "items": { "$ref": "#/definitions/Interface" } }, "service": { "title": "Service", "default": [], "type": "array", "items": { "$ref": "#/definitions/Service" } }, "ssh_keys": { "title": "Ssh Keys", "default": [], "type": "array", "items": { "$ref": "#/definitions/SSHKey" } }, "related": { "title": "Related", "default": { "emails": [], "files": [], "hash": [], "hosts": [], "ip": [], "mac": [], "ports": [], "protocols": [], "process": [], "roles": [], "urls": [], "user": [] }, "allOf": [ { "$ref": "#/definitions/Related" } ] }, "registers": { "title": "Registers", "default": [], "type": "array", "items": { "$ref": "#/definitions/Register" } }, "tag": { "title": "Tag", "default": [], "type": "array", "items": { "$ref": "#/definitions/Tag" } }, "io": { "title": "Io", "default": [], "type": "array", "items": { "$ref": "#/definitions/IO" } }, "event": { "title": "Event", "default": [], "type": "array", "items": { "$ref": "#/definitions/Event" } }, "memory": { "title": "Memory", "default": [], "type": "array", "items": { "$ref": "#/definitions/Memory" } }, "module": { "title": "Module", "default": [], "type": "array", "items": { "$ref": "#/definitions/DeviceData" } }, "users": { "title": "Users", "default": [], "type": "array", "items": { "$ref": "#/definitions/User" } }, "x509": { "title": "X509", "default": { "alternative_names": [], "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "issuer": { "common_name": "", "country": "", "distinguished_name": "", "locality": "", "organization": "", "organizational_unit": "", "state_or_province": "" }, "not_after": null, "not_before": null, "original": "", "public_key_algorithm": "", "public_key_curve": "", "public_key_exponent": null, "public_key_size": null, "serial_number": "", "signature_algorithm": "", "subject": { "common_name": "", "country": "", "distinguished_name": "", "locality": "", "organization": "", "organizational_unit": "", "state_or_province": "" }, "version_number": "" }, "allOf": [ { "$ref": "#/definitions/X509" } ] }, "uefi_image": { "title": "Uefi Image", "default": [], "type": "array", "items": { "$ref": "#/definitions/UEFIFile" } }, "uefi_hashes": { "title": "Uefi Hashes", "default": [], "type": "array", "items": { "$ref": "#/definitions/UEFIHash" } }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false } } }
- Fields:
- Validators:
-
description:
Description¶ Identifying information such as vendor, brand, and model.
-
endian:
Literal['big','little','']¶ “Endianness” of the CPU of the system where the memory was read from.
-
hardware:
Hardware¶ Information about the device’s hardware specifications and configuration (RAM, storage, etc.).
-
hostname:
ConstrainedStrValue¶ Hostname of the device (if resolved). In the case of a device with multiple communication modules, this is the hostname of the module PEAT primarily uses to communicate (or first discovered).
-
id:
ConstrainedStrValue¶ Unique identifier for the device. Can be anything, as long as it’s consistent in the module. Defaults to the device MAC, IP, or COM port.
-
ip:
str¶ IP address of the device. In the case of a device with multiple communication modules, this is the IP address of the module PEAT primarily uses to communicate (or first discovered).
- Validated by:
validate_ip
-
mac:
str¶ MAC address of the device. In the case of a device with multiple communication modules, this is the MAC address of the module PEAT primarily uses to communicate (or first discovered).
- Validated by:
validate_mac
-
mac_vendor:
str¶ Vendor name resolved from the MAC address OUI.
This field is auto-populated by PEAT if the
macfield is set.
-
serial_port:
ConstrainedStrValue¶ Serial port on the local system connected to the device. This could be a Windows COM port, e.g.
COM4, or a Linux file path, e.g./dev/ttyS0. This is also used for USB connections. To get the specific serial settings, lookup the interface with the matching port indata.interface.
-
name:
ConstrainedStrValue¶ Name to refer to the device as, e.g. as pulled from a config or resolved via DNS. Defaults to FQDN resolved from the IP address, if hostname resolutions are enabled in the PEAT configuration.
-
part_number:
ConstrainedStrValue¶ Part number of the device, as defined by the vendor and stored on the device.
-
type:
ConstrainedStrValue¶ The type/class of device, e.g. “PLC”, “Relay”, “RTU”, “Controller” (catch-all), etc. Examples of type for a module include Communications Adapter, General Purpose Discrete I/O, or CPU.
-
serial_number:
ConstrainedStrValue¶ Unique serial number of the device, as defined by the vendor and stored on the device.
-
run_mode:
ConstrainedStrValue¶ Run mode of the device. For example, on a PLC, there may be a key in the front of the device that sets PROG or RUN (program vs running). What this field means depends on the device, for instance a PLC’s potential run modes will differ from a RTU’s potential run modes.
-
slot:
ConstrainedStrValue¶ Position of the device in a rack or larger device. This can be a relative position, e.g. “0” for the first module in a PLC, or a name or other identifier for the position (such as an internal bus address).
-
status:
ConstrainedStrValue¶ Status of the device. The meaning of this field is device-dependant.
-
uptime:
timedelta|None¶ Number of seconds the host has been up (powered on/online), as either a integer or
timedelta.
-
geo:
Geo¶ Geolocation information. This includes the device’s physical location and configured timezone.
Information that is related to a host or interface.
-
registers:
list[Register]¶ All Input/Output (I/O) protocol data points configured on the device, e.g. DNP3 and Modbus.
-
tag:
list[Tag]¶ Data variables (“tags”) in a device. Often mapped in a device’s logic to physical I/O and/or registers.
-
module:
list[DeviceData]¶ Physical add-on modules in a device, e.g. slots in a PLC or rack. These also include add-on components, such as a wireless radio. These can include analog and digital I/O modules, COMMs modules (Ethernet, various serial protocols, Wi-Fi, LTE, etc.), CPU modules, and anything else really. While there are general sorts of modules that are typically seen in devices like a PLC, the reality is there are a ton of modules that sometimes highly specific to a vendor or application. Therefore, while we define a set of module types, they are not required to be used if the module does not fall in the set of defined types.
-
x509:
X509¶ x509 certificate associated with the device, e.g. from a HTTPS/TLS service or extracted from a file in a blob.
-
uefi_hashes:
list[UEFIHash]¶ uefi_hashes holds all the hashes for a file that is linked to a device.
-
extra:
dict¶ Additional vendor/model-specific information that doesn’t currently fit into the defined model, but may be useful and we don’t want to leave on the cutting room floor, so to speak. In other words: a piece of data belongs here if it’s useful and doesn’t fit elsewhere in the model.
-
label:
str¶ User-specified label from the PEAT configuration file.
This field is automatically set by PEAT, and device modules shouldn’t write to this field.
-
comment:
str¶ User-specified comment from the PEAT configuration file.
This field is automatically set by PEAT, and device modules shouldn’t write to this field.
- property options: DeepChainMap¶
PEAT configuration options for this device. This includes Service configurations (timeout, port, etc.), login credentials, etc.
The options are composed from multiple sources and use the following order of precedence:
Runtime changes (
self._runtime_options)Host-specific changes (
self._host_option_overrides)Global option changes (stored in
datastore.global_options)Module-specific defaults (from
peat.device.DeviceModule.default_optionsfor the module)Global defaults (
peat.data.DeviceData._DEFAULT_OPTIONS)
- get_id(attribute_precedence=None)[source]¶
Get a canonical device ID.
If the lookup fails a randomly generated ID is used, generated by
gen_random_dev_id().The attribute used as the ID is selected based on an order of precedence. Each attribute is checked in the order defined, and the first attribute with a defined value (non-empty) is used as the ID.
- get_comm_id()[source]¶
Get a canonical communication protocol ID for this device (e.g. IP address, MAC address, serial port)
Same as
get_id()except_COMM_ID_KEY_ORDERis used as the order of precedence.- Return type:
- Returns:
Communication protocol ID of the device (e.g. IP, MAC, serial port)
- service_status(lookup)[source]¶
Returns the status of a service, or
"unknown"if the service isn’t found.- Return type:
- export(include_original=False, exclude_fields=None, only_fields=None)[source]¶
Return device data as a normalized JSON-friendly
dict.- Parameters:
include_original (
bool) -- Iforiginalkeys should be included (this is the raw data, e.g. raw firmware or raw logic)exclude_fields (
list[str] |None) -- Field names (keys) to exclude from the returneddict. This recursively excludes fields!only_fields (
str|list[str] |None) -- Only include the specified fields (keys) in the returneddict(Note: this only applies to top-level fields in the data, e.g.name,firmware, etc.)
- Return type:
- Returns:
The exported data as a JSON-serializable
dictNote
The order of data returned will be the same as the order of the fields in the models
- export_summary(cached_export=None)[source]¶
Return a summarized version of the device data as a normalized JSON-friendly
dict, with certain large fields removed.
- export_to_files(overwrite_existing=False)[source]¶
Export data to files named
device-data-fullanddevice-data-summary.
- elastic()[source]¶
This generates the
hostportion of Elasticsearch data.Note
Attributes in any data objects with an empty value or a name that starts with an underscore (
_) will not be included in the return value
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- write_file(data, filename, overwrite_existing=False, out_dir=None, merge_existing=False)[source]¶
Save data to a file in the device’s output directory.
Note
Data will NOT be written if both
DEVICE_DIRandself._out_dirare unset.- Parameters:
data (
Any) -- Raw data to writefilename (
str) -- Name including extension of the fileoverwrite_existing (
bool) -- If existing files with the same name should be overwritten instead of being written with a “.<num>” appended to the name.out_dir (
Path|None) -- Directory the data should be written to. Defaults to result ofdev.get_out_dir()merge_existing (
bool) -- If the file already exists and is JSON, then read the data from the existing file, merge the new data with it, then overwrite the file with the merged data.
- Return type:
- Returns:
Path to the file that was written
- get_out_dir()[source]¶
Get the path to the directory for any file output and results associated with this device.
- Return type:
- get_sub_dir(basename)[source]¶
Generate a directory path for specific file output, for example FTP files. The path will be a sub-dir in the device’s results directory.
- Return type:
- populate_fields(network_only=False)[source]¶
Populate new values by extrapolating from other existing values.
Note
This method also removes duplicate services and interfaces
Example: if the device object only has a
macfield populated, this will resolve and update theipandhostnamefields, then add aethernetInterfacewith those fields populated.
- retrieve(attr, search)[source]¶
Retrieve a complex device data value.
>>> from peat.data import DeviceData, Interface, Service, Tag >>> dev = DeviceData() >>> dev.store("interface", Interface(ip="192.0.2.123", type="ethernet")) >>> dev.store("interface", Interface(ip="192.0.2.20", type="ethernet")) >>> dev.store("service", Service(protocol="http", port=80)) >>> dev.store("tag", Tag(name="var_rtu-8_I0", type="binary")) >>> dev.store("tag", Tag(name="var_rtu-9_I1", type="binary")) >>> dev.store("tag", Tag(name="var_rtu-10_Q0", type="analog")) # Interface with IP address of 192.0.2.20 >>> iface = dev.retrieve("interface", {"ip": "192.0.2.20"}) >>> iface.ip '192.0.2.20' # All "ethernet" interfaces >>> eth_ifaces = dev.retrieve("interface", {"type": "ethernet"}) >>> len(eth_ifaces) 2 >>> iface in eth_ifaces True # The 'HTTP' service >>> svc = dev.retrieve("service", {"protocol": "http"}) >>> svc.port 80 # Tag with name of var_rtu-8_I0 >>> tag = dev.retrieve("tag", {"name": "var_rtu-8_I0"}) >>> tag.name 'var_rtu-8_I0' # All the binary tags >>> binary_tags = dev.retrieve("tag", {"type": "binary"}) >>> len(binary_tags) 2 >>> tag in binary_tags True
- store(key, value, lookup=None, interface_lookup=None, append=False)[source]¶
Add or update complex device data.
>>> from datetime import datetime >>> from pprint import pprint >>> from peat.data import DeviceData, Interface, Memory, Tag, Register # Create the device instance >>> dev = DeviceData() # Add a single network interface with IP of 192.0.2.20 # NOTE: MAC address and hostname will be auto-resolved # the next time "dev.populate_fields()" is called. >>> dev.store("interface", Interface(ip="192.0.2.20", type="ethernet")) >>> dev.export(only_fields="interface") {'interface': [{'type': 'ethernet', 'ip': '192.0.2.20'}]} # Add a HTTP service to the interface with an IP of 192.0.2.20 >>> dev.store( "service", Service(protocol="http", port=80), # Lookup the interface for the service to be associated with interface_lookup={"ip": "192.0.2.20"}) >>> dev.export(only_fields="service") {'service': [{'port': 80, 'protocol': 'http', 'transport': 'tcp'}]} >>> pprint(dev.export(only_fields="interface")) {'interface': [{'ip': '192.0.2.20', 'services': [{'port': 80, 'protocol': 'http', 'transport': 'tcp'}], 'type': 'ethernet'}]} # Services are also stored in interfaces >>> dev.service[0] == dev.interface[0].services[0] True # However, keep in mind it's not the same instance, so changes to the # interface in dev.service will not be reflected in the one in # interface.services. If making changes, use store(). >>> dev.service[0] is dev.interface[0].services[0] False # I/O protocol registers, e.g. for Modbus and DNP3 >>> dev.store("registers", Register(protocol="dnp3", data_type="bool")) >>> pprint(dev.export(only_fields="registers")) {'registers': [{'data_type': 'bool', 'protocol': 'dnp3'}]} # I/O tags, e.g. from a SCADA database >>> dev.store("tag", Tag(name="var_rtu-8_I0", type="binary")) >>> pprint(dev.export(only_fields="tag")) {'tag': [{'name': 'var_rtu-8_I0', 'type': 'binary'}]} # Store a raw read from device memory >>> dev.store("memory", Memory( address="0000FFAB", created=datetime(2019, 2, 25, 17, 39, 11, 507318), value="D3ADB33F")) >>> dev.memory [Memory(address='0000FFAB', created=datetime.datetime(2019, 2, 25, 17, 39, 11, 507318), device='192.0.2.20', value='D3ADB33F')] >>> pprint(dev.export(only_fields="memory")) {'memory': [{'address': '0000FFAB', 'created': '2019-02-25 17:39:11.507318', 'device': '192.0.2.20', 'value': 'D3ADB33F'}]} # Adding a module by constructing a new DeviceData object >>> io_module = DeviceData(name="digitalIO", type="I/O", slot="1") >>> dev.store("module", io_module) >>> dev.export(only_fields="module") {'module': [{'name': 'digitalIO', 'type': 'I/O', 'slot': '1'}]}
Note
If unset, the
idattribute on this object will be set to the IP of the first Interface added viastore()Note
When adding a service, the interface the service should be associated with can be specified by including specific keys in the
interface_lookupargument. These keys are:name,ip,serial_port,mac, andhostname. Example:interface_lookup={"ip": "192.0.2.20"}will add the service to the Interface object with an IP address of192.0.2.20.- Parameters:
key (
Literal['interface','service','ssh_keys','registers','tag','io','event','memory','module','users','uefi_image','uefi_hashes','files']) -- Name of the field to add or edit, e.g.interfaceto add data to a new or existing interface.value (
BaseModel) -- Value to store. Type and structure depends on the field being changed.lookup (
str|list|dict|None) --Values to use to search for an existing item to edit.
Note
If
None, thenlookupwill fallback to hardcoded search defaults if the type isServiceorInterface.The lookup value can be one of the following:
- String of an attribute name to compare, e.g.
"ip" to use the
ipattribute to compare interfaces.
- String of an attribute name to compare, e.g.
- A list of strings of attribute names to compare, e.g.
["name", "ip"]. The attributes will be checked in order, so a interface with the samenameattribute will be merged before one that matches theipattribute.
- a dict of values to lookup, with key being attribute name
and value the value to compare. ALL values MUST match for a lookup to be successful!
Examples of different lookup argument data types¶>>> from pprint import pprint >>> from peat.data import DeviceData, Memory, Service, IO >>> dev = DeviceData(ip="192.0.2.20") # Specify name of a service to update >>> dev.store("service", Service(protocol="telnet")) >>> dev.export(only_fields="service") {'service': [{'protocol': 'telnet', 'transport': 'tcp'}]} >>> dev.store("service", value=Service(status="open"), lookup={"protocol": "telnet"}) >>> dev.export(only_fields="service") {'service': [{'protocol': 'telnet', 'status': 'open', 'transport': 'tcp'}]} # Lookup using a key >>> dev.store("memory", Memory(address="0000FFAB")) >>> dev.export(only_fields="memory") {'memory': [{'address': '0000FFAB', 'device': '192.0.2.20'}]} >>> dev.store("memory", value=Memory( address="0000FFAB", created=datetime(2019, 2, 25, 17, 39, 11, 507318), ), lookup="address") >>> pprint(dev.export(only_fields="memory")) {'memory': [{'address': '0000FFAB', 'created': '2019-02-25 17:39:11.507318', 'device': '192.0.2.20'}]} # Lookup using list of keys >>> dev.store("io", IO(address="0001", direction="input")) >>> dev.export(only_fields="io") {'io': [{'address': '0001', 'direction': 'input'}]} >>> dev.store("io", IO(address="0001", direction="input", type="analog"), lookup=["address", "direction"] ) >>> dev.export(only_fields="io") {'io': [{'address': '0001', 'direction': 'input', 'type': 'analog'}]}
interface_lookup (
dict|None) --dictwithInterfaceattribute keys and values to lookup when storing aServiceappend (
bool) -- Append the item to the list and don’t attempt lookups
- Raises:
PeatError -- Invalid key specified or other errors indicative of issues with module code
- Return type:
- is_duplicate(other)[source]¶
If this device is likely a duplicate of another.
Note
Only deduplicate if devices have the same communication ID (IP, MAC, Serial port) or label (from a PEAT config file)
- Parameters:
other (
DeviceData) -- Device to compare- Return type:
- Returns:
If the device is likely a duplicate of this device
- purge_duplicates(force=False)[source]¶
Removes duplicates from all
list-type attributes on this object that aren’t private.Once performed,
self._is_deduplicatedis set to True. If True, subsequent calls won’t perform deduplication. To override this behavior, set force=True, or setself._is_deduplicatedto False.- Return type:
8.1.3. Data Models¶
Note
Most fields with a type of peat.data.models.ConstrainedStrValue are just str type, but will automatically have any whitespace stripped when assigned to.
Identifying information such as vendor, brand, and model. |
|
Device log entry, such as logins, metering reads, or system events. |
|
Contextual information and metadata for a file. |
|
Device firmware. |
|
Geolocation information (the device's physical location). |
|
Hardware information of the device, e.g. amount of RAM. |
|
Hashes of raw data or a file. |
|
Physical Input/Output (I/O) connections on a device. |
|
Communication interface, such as a Ethernet port or Serial link. |
|
Latitude and Longitude (geographical coordinates). |
|
What the device has been programmed to do (it's "logic"). |
|
Physical memory values (e.g. RAM, EEPROM). |
|
Operating System (OS) information, such as the name and version. |
|
Configured I/O protocol data point ("registers"), e.g. DNP3 or Modbus/TCP. |
|
Information that is related to a device or interface, or was found on the device. |
|
Communication protocol "service" configured or running on the device. |
|
SSH keys (public or private). |
|
Identifies a device vendor (SEL, Schneider Electric, Siemens, etc). |
|
x509 certificate. |
|
Issuer or Subject in a x509 certificate. |
- Vendor[source]¶
Identifies a device vendor (SEL, Schneider Electric, Siemens, etc).
Show JSON schema
{ "title": "Vendor", "description": "Identifies a device vendor (SEL, Schneider Electric, Siemens, etc).", "type": "object", "properties": { "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" } }, "additionalProperties": false }
-
id:
ConstrainedStrValue¶ Abbreviated version of the vendor name that can be used for lookups.
Examples
SELWindRiverSchneiderSiemensSandia
-
name:
ConstrainedStrValue¶ The full expanded vendor name. Used for display in a visualization or dashboard.
Examples
Schweitzer Engineering LaboratoriesWind River SystemsSchneider Electric
-
id:
- Description[source]¶
Identifying information such as vendor, brand, and model.
Show JSON schema
{ "title": "Description", "description": "Identifying information such as vendor, brand, and model.", "type": "object", "properties": { "brand": { "title": "Brand", "default": "", "type": "string" }, "contact_info": { "title": "Contact Info", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "full": { "title": "Full", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "model": { "title": "Model", "default": "", "type": "string" }, "product": { "title": "Product", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "vendor": { "title": "Vendor", "default": { "id": "", "name": "" }, "allOf": [ { "$ref": "#/definitions/Vendor" } ] } }, "additionalProperties": false, "definitions": { "Vendor": { "title": "Vendor", "description": "Identifies a device vendor (SEL, Schneider Electric, Siemens, etc).", "type": "object", "properties": { "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" } }, "additionalProperties": false } } }
- Fields:
- Validators:
strip_quotes»contact_infostrip_quotes»description
-
brand:
ConstrainedStrValue¶ Brand of the device. Can be empty string if not applicable, such as for most SEL devices.
Examples
ModiconPowerLogic ION""
-
contact_info:
str¶ Contact info for the device, e.g. an email address, name, or phone number. This is commonly retrieved from SNMP.
- Validated by:
strip_quotes
-
description:
str¶ Free-form description of the device, such as a “description” configuration value extracted from the device or other general information that is useful to note.
- Validated by:
strip_quotes
-
full:
ConstrainedStrValue¶ Combination of vendor, brand, model, and any other identifiers. Used to perform lookups with fuzzy string matching.
Examples
Schneider Electric Modicon M340SEL-351S
-
model:
ConstrainedStrValue¶ Model of the device.
Examples
M340351S
-
product:
ConstrainedStrValue¶ The product identifier for the device, minus the vendor. This is includes the brand and model.
Examples
Modicon M340351S
- Hardware[source]¶
Hardware information of the device, e.g. amount of RAM.
Show JSON schema
{ "title": "Hardware", "description": "Hardware information of the device, e.g. amount of :term:`RAM`.", "type": "object", "properties": { "cpu": { "title": "Cpu", "default": { "brand": "", "contact_info": "", "description": "", "full": "", "model": "", "product": "", "vendor": { "id": "", "name": "" } }, "allOf": [ { "$ref": "#/definitions/Description" } ] }, "id": { "title": "Id", "default": "", "type": "string" }, "storage_available": { "title": "Storage Available", "minimum": 0, "type": "integer" }, "storage_usage": { "title": "Storage Usage", "minimum": 0, "type": "integer" }, "storage_total": { "title": "Storage Total", "minimum": 0, "type": "integer" }, "storage_type": { "title": "Storage Type", "default": "", "type": "string" }, "memory_available": { "title": "Memory Available", "minimum": 0, "type": "integer" }, "memory_usage": { "title": "Memory Usage", "minimum": 0, "type": "integer" }, "memory_total": { "title": "Memory Total", "minimum": 0, "type": "integer" }, "memory_type": { "title": "Memory Type", "default": "", "type": "string" }, "revision": { "title": "Revision", "default": "", "type": "string" }, "version": { "title": "Version", "default": "", "type": "string" } }, "additionalProperties": false, "definitions": { "Vendor": { "title": "Vendor", "description": "Identifies a device vendor (SEL, Schneider Electric, Siemens, etc).", "type": "object", "properties": { "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" } }, "additionalProperties": false }, "Description": { "title": "Description", "description": "Identifying information such as vendor, brand, and model.", "type": "object", "properties": { "brand": { "title": "Brand", "default": "", "type": "string" }, "contact_info": { "title": "Contact Info", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "full": { "title": "Full", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "model": { "title": "Model", "default": "", "type": "string" }, "product": { "title": "Product", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "vendor": { "title": "Vendor", "default": { "id": "", "name": "" }, "allOf": [ { "$ref": "#/definitions/Vendor" } ] } }, "additionalProperties": false } } }
- Fields:
-
cpu:
Description¶ Information about the CPU on the device, such as the vendor and model.
-
id:
ConstrainedStrValue¶ Hardware ID of the device.
-
storage_available:
ConstrainedIntValue|None¶ Amount of persistent storage currently available on the device, in bytes.
- Constraints:
minimum = 0
-
storage_usage:
ConstrainedIntValue|None¶ Amount of persistent storage currently in use on the device, in bytes.
- Constraints:
minimum = 0
-
storage_total:
ConstrainedIntValue|None¶ Total amount of storage on the device, in bytes.
- Constraints:
minimum = 0
-
storage_type:
ConstrainedStrValue¶ Type of storage on the device.
Values should be lowercase and underscore-separated.
Examples
hddssdnvram
-
memory_available:
ConstrainedIntValue|None¶ Amount of volatile memory (e.g. RAM) currently available, in bytes.
- Constraints:
minimum = 0
-
memory_usage:
ConstrainedIntValue|None¶ Amount of volatile memory (e.g. RAM) currently in use, in bytes.
- Constraints:
minimum = 0
-
memory_total:
ConstrainedIntValue|None¶ Total amount of volatile memory (e.g. RAM) on the device, in bytes.
- Constraints:
minimum = 0
-
memory_type:
ConstrainedStrValue¶ Type of volatile memory on the device, lowercase and underscore-separated.
Examples
ddr2_sdram
-
revision:
ConstrainedStrValue¶ Hardware revision of the device (e.g. MinorRev field in Rockwell L5X). This is distinct from the software (e.g., firmware or OS version), and is purely for the hardware itself (e.g., the mainboard or module). The detailed meaning of the value in this field is device-dependant.
-
version:
ConstrainedStrValue¶ Hardware version of the device (e.g. MajorRev field in Rockwell L5X). This is distinct from the software (e.g., firmware or OS version), and is purely for the hardware itself (e.g., the mainboard or module). The detailed meaning of the value in this field is device-dependant.
- Hash[source]¶
Hashes of raw data or a file.
Note
All hashes are uppercase hexadecimal strings, per ECS
Show JSON schema
{ "title": "Hash", "description": "Hashes of raw data or a file.\n\n.. note::\n All hashes are uppercase hexadecimal strings, per :term:`ECS`", "type": "object", "properties": { "md5": { "title": "MD5 hash", "minLength": 32, "maxLength": 32, "type": "string" }, "sha1": { "title": "SHA1 hash", "minLength": 40, "maxLength": 40, "type": "string" }, "sha256": { "title": "SHA256 hash", "minLength": 64, "maxLength": 64, "type": "string" }, "sha512": { "title": "SHA512 hash", "minLength": 128, "maxLength": 128, "type": "string" } }, "additionalProperties": false }
- Fields:
- Validators:
-
md5:
ConstrainedStrValue|None¶ MD5 hash.
- Constraints:
minLength = 32
maxLength = 32
- Validated by:
validate_hash
-
sha1:
ConstrainedStrValue|None¶ SHA1 hash.
- Constraints:
minLength = 40
maxLength = 40
- Validated by:
validate_hash
- User[source]¶
Information describing a user on a device.
Show JSON schema
{ "title": "User", "description": "Information describing a user on a device.", "type": "object", "properties": { "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "domain": { "title": "Domain", "default": "", "type": "string" }, "email": { "title": "Email", "default": "", "type": "string" }, "full_name": { "title": "Full Name", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "permissions": { "title": "Permissions", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "roles": { "title": "Roles", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "uid": { "title": "Uid", "default": "", "type": "string" }, "gid": { "title": "Gid", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false }
- Fields:
-
description:
ConstrainedStrValue¶ General description of the user (this is open to interpretation).
-
domain:
ConstrainedStrValue¶ Name of the domain the user is a member of.
For example, an LDAP or Active Directory domain name.
-
email:
ConstrainedStrValue¶ User email address.
Examples
-
full_name:
ConstrainedStrValue¶ The user’s full name, if known.
Examples
Billy Bob Joe
Administrator
-
name:
ConstrainedStrValue¶ Short name or login of the user.
-
uid:
ConstrainedStrValue¶ The user’s numeric user ID, if applicable.
-
gid:
ConstrainedStrValue¶ The user’s numeric group ID, if applicable.
- Related[source]¶
Information that is related to a device or interface, or was found on the device.
Show JSON schema
{ "title": "Related", "description": "Information that is related to a device or interface, or was found on the device.", "type": "object", "properties": { "emails": { "title": "Emails", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "files": { "title": "Files", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "hash": { "title": "Hash", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "hosts": { "title": "Hosts", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "ip": { "title": "Ip", "default": [], "elastic_type": "ip", "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "mac": { "title": "Mac", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "ports": { "title": "Ports", "default": [], "type": "array", "items": { "type": "integer", "minimum": 1, "maximum": 65535 }, "uniqueItems": true }, "protocols": { "title": "Protocols", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "process": { "title": "Process", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "roles": { "title": "Roles", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "urls": { "title": "Urls", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "user": { "title": "User", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true } }, "additionalProperties": false }
- Fields:
- Validators:
-
files:
set[ConstrainedStrValue]¶ Any files found on the device or referenced from the device’s configuration. These can either be absolute paths or just filenames. Absolute paths are preferred, if known.
Examples
/home/user/config.txtconfig.xml
-
hash:
set[str]¶ Hashes related to the device or interface.
Allowed hash types: MD5, SHA1, SHA256, SHA512.
- Validated by:
validate_hash
-
ip:
set[str]¶ IP addresses related to the device or interface. These can be IPv4 or IPv6 addresses.
- Validated by:
validate_ip
-
protocols:
set[str]¶ Application layer (OSI Layer 7) protocols related to a device or interface.
Values should be lowercase, underscore-separated, with no whitespace.
The format is the same as the
protocolfield inInterface.Examples
modbus_tcpdnp3ftp
-
process:
set[str]¶ Names of processes that are currently running on the device and/or ran at some point in the device’s history (e.g. obtained from a log file).
Examples:
telnetd
- File[source]¶
Contextual information and metadata for a file.
The file could be on disk, in memory, a directory, or simply represent an artifact that’s known to be on the device but PEAT doesn’t have the ability to access.
Show JSON schema
{ "title": "File", "description": "Contextual information and metadata for a file.\n\nThe file could be on disk, in memory, a directory, or simply\nrepresent an artifact that's known to be on the device but\nPEAT doesn't have the ability to access.", "type": "object", "properties": { "created": { "title": "Created", "type": "string", "format": "date-time" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "device": { "title": "Device", "default": "", "type": "string" }, "directory": { "title": "Directory", "default": "", "type": "string" }, "extension": { "title": "Extension", "default": "", "type": "string" }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "local_path": { "title": "Local Path", "type": "string", "format": "path" }, "path": { "title": "Path", "type": "string", "format": "path" }, "peat_module": { "title": "Peat Module", "default": "", "type": "string" }, "gid": { "title": "Gid", "default": "", "type": "string" }, "group": { "title": "Group", "default": "", "type": "string" }, "mime_type": { "title": "Mime Type", "default": "", "type": "string" }, "mode": { "title": "Mode", "default": "", "type": "string" }, "mtime": { "title": "Mtime", "type": "string", "format": "date-time" }, "name": { "title": "Name", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "type": "string", "format": "binary" }, "owner": { "title": "Owner", "default": "", "type": "string" }, "size": { "title": "Size", "minimum": 0, "type": "integer" }, "target_path": { "title": "Target Path", "type": "string", "format": "path" }, "type": { "title": "Type", "default": "", "enum": [ "file", "dir", "symlink", "" ], "type": "string" }, "uid": { "title": "Uid", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false, "definitions": { "Hash": { "title": "Hash", "description": "Hashes of raw data or a file.\n\n.. note::\n All hashes are uppercase hexadecimal strings, per :term:`ECS`", "type": "object", "properties": { "md5": { "title": "MD5 hash", "minLength": 32, "maxLength": 32, "type": "string" }, "sha1": { "title": "SHA1 hash", "minLength": 40, "maxLength": 40, "type": "string" }, "sha256": { "title": "SHA256 hash", "minLength": 64, "maxLength": 64, "type": "string" }, "sha512": { "title": "SHA512 hash", "minLength": 128, "maxLength": 128, "type": "string" } }, "additionalProperties": false } } }
- Fields:
- Validators:
convert_arbitrary_path_to_purepath»pathconvert_arbitrary_path_to_purepath»target_path
-
description:
ConstrainedStrValue¶ General human-readable description of what the file is.
-
device:
ConstrainedStrValue¶ Device that is the source of the file. If this is a static parse, then it should be the name of the system the file was recovered from or parsed on. Otherwise, this should be the ID of the device it was pulled from.
-
directory:
ConstrainedStrValue¶ Path to the directory where the file is located.
-
extension:
ConstrainedStrValue¶ File extension, without a leading
.character.Examples
txttar.gzxmlzip
- Validated by:
-
local_path:
Path|None¶ Concrete path of the file on the local system (the system running PEAT).
- Constraints:
type = string
format = path
-
path:
PurePath|None¶ Path of the file, in it’s original form. This may be either the path to the file on the device, or the path from the system it originated from (e.g. as extracted from a project file or using PEAT Pillage).
- Constraints:
type = string
format = path
- Validated by:
convert_arbitrary_path_to_purepath
-
peat_module:
ConstrainedStrValue¶ PEAT module associated with this file artifact.
-
gid:
ConstrainedStrValue¶ Primary group ID (GID) of the file.
-
group:
ConstrainedStrValue¶ File’s owning group name.
-
mime_type:
ConstrainedStrValue¶ MIME type should identify the format of the file or stream of bytes using the IANA official types, where possible. When more than one type is applicable, the most specific type should be used.
-
mode:
ConstrainedStrValue¶ Mode of the file in octal representation.
Examples:
064006440777
-
name:
ConstrainedStrValue¶ File’s name, including extension (e.g.
SET_ALL.txt).
-
owner:
ConstrainedStrValue¶ File owner’s username.
-
size:
ConstrainedIntValue|None¶ Size of the file in bytes.
Only relevant when
file.typeis"file".- Constraints:
minimum = 0
-
target_path:
PurePath|None¶ Target path for symlinks.
Only relevant when
file.typeis"symlink".- Constraints:
type = string
format = path
- Validated by:
convert_arbitrary_path_to_purepath
-
type:
Literal['file','dir','symlink','']¶ File type, following the ECS.
Allowed values
file: It’s a filedir: It’s a directorysymlink: It’s a symbolic link
-
uid:
ConstrainedStrValue¶ Numeric user ID (UID) or security identifier (SID) of the file owner.
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- Firmware[source]¶
Device firmware.
Show JSON schema
{ "title": "Firmware", "description": "Device firmware.", "type": "object", "properties": { "checksum": { "title": "Checksum", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "file": { "title": "File", "default": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "allOf": [ { "$ref": "#/definitions/File" } ] }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "id": { "title": "Id", "default": "", "type": "string" }, "last_updated": { "title": "Last Updated", "type": "string", "format": "date-time" }, "original": { "title": "Original", "default": "", "type": "string", "format": "binary" }, "revision": { "title": "Revision", "default": "", "type": "string" }, "release_date": { "title": "Release Date", "type": "string", "format": "date-time" }, "timestamp": { "title": "Timestamp", "type": "string", "format": "date-time" }, "version": { "title": "Version", "default": "", "type": "string" } }, "additionalProperties": false, "definitions": { "Hash": { "title": "Hash", "description": "Hashes of raw data or a file.\n\n.. note::\n All hashes are uppercase hexadecimal strings, per :term:`ECS`", "type": "object", "properties": { "md5": { "title": "MD5 hash", "minLength": 32, "maxLength": 32, "type": "string" }, "sha1": { "title": "SHA1 hash", "minLength": 40, "maxLength": 40, "type": "string" }, "sha256": { "title": "SHA256 hash", "minLength": 64, "maxLength": 64, "type": "string" }, "sha512": { "title": "SHA512 hash", "minLength": 128, "maxLength": 128, "type": "string" } }, "additionalProperties": false }, "File": { "title": "File", "description": "Contextual information and metadata for a file.\n\nThe file could be on disk, in memory, a directory, or simply\nrepresent an artifact that's known to be on the device but\nPEAT doesn't have the ability to access.", "type": "object", "properties": { "created": { "title": "Created", "type": "string", "format": "date-time" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "device": { "title": "Device", "default": "", "type": "string" }, "directory": { "title": "Directory", "default": "", "type": "string" }, "extension": { "title": "Extension", "default": "", "type": "string" }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "local_path": { "title": "Local Path", "type": "string", "format": "path" }, "path": { "title": "Path", "type": "string", "format": "path" }, "peat_module": { "title": "Peat Module", "default": "", "type": "string" }, "gid": { "title": "Gid", "default": "", "type": "string" }, "group": { "title": "Group", "default": "", "type": "string" }, "mime_type": { "title": "Mime Type", "default": "", "type": "string" }, "mode": { "title": "Mode", "default": "", "type": "string" }, "mtime": { "title": "Mtime", "type": "string", "format": "date-time" }, "name": { "title": "Name", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "type": "string", "format": "binary" }, "owner": { "title": "Owner", "default": "", "type": "string" }, "size": { "title": "Size", "minimum": 0, "type": "integer" }, "target_path": { "title": "Target Path", "type": "string", "format": "path" }, "type": { "title": "Type", "default": "", "enum": [ "file", "dir", "symlink", "" ], "type": "string" }, "uid": { "title": "Uid", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false } } }
- Fields:
-
checksum:
ConstrainedStrValue¶ Checksum used by the device to verify the firmware image is valid. This is usually found in or with the firmware image file or the device configuration.
-
extra:
dict¶ Additional unstructured information related to the firmware, generally vendor-specific information such as “settings_version”.
-
hash:
Hash¶ Hashes of the raw firmware (the contents of
original).Note
This may differ from the file hash, if present.
-
id:
ConstrainedStrValue¶ Firmware identification string, e.g. the “FID” or “BFID” strings in SEL devices.
-
revision:
ConstrainedStrValue¶ Revision of the device’s current firmware (or operating system). Common field seen on devices that’s sometimes distinct from the canonical version string.
-
timestamp:
datetime|None¶ Timestamp as extracted from the device or firmware, device-dependent meaning. Often represents when the firmware was compiled/built or released.
-
version:
ConstrainedStrValue¶ Version of the device’s current firmware (or operating system).
- Logic[source]¶
What the device has been programmed to do (it’s “logic”).
In a PLC, the logic is one or more of the five IEC 61181-3 languages:
Ladder Diagram (LD)
Function Block Diagram (FBD)
Structured Text (ST)
Instruction List (IL)
Sequential Function Chart (SFC)
In a Relay, the logic is the protection schemes.
In a Power Meter, the logic is the programmed metering/monitoring setpoints.
Note
Logic is separate from protocol register mappings or values, such as Modbus or DNP3, as well as memory values. There is sometimes overlap, as some devices have been known to store their logic as e.g. a set of Modbus registers.
Show JSON schema
{ "title": "Logic", "description": "What the device has been programmed to do (it's \"logic\").\n\nIn a :term:`PLC`, the logic is one or more of the five `IEC 61181-3\n<https://en.wikipedia.org/wiki/IEC_61131-3>`_ languages:\n\n- Ladder Diagram (LD)\n- Function Block Diagram (FBD)\n- Structured Text (ST)\n- Instruction List (IL)\n- Sequential Function Chart (SFC)\n\nIn a Relay, the logic is the protection schemes.\n\nIn a Power Meter, the logic is the programmed metering/monitoring setpoints.\n\n.. note::\n Logic is separate from protocol register mappings or values, such as\n Modbus or DNP3, as well as memory values. There is sometimes overlap,\n as some devices have been known to store their logic as e.g. a set\n of Modbus registers.", "type": "object", "properties": { "author": { "title": "Author", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "created": { "title": "Created", "type": "string", "format": "date-time" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "file": { "title": "File", "default": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "allOf": [ { "$ref": "#/definitions/File" } ] }, "formats": { "title": "Formats", "default": {}, "elastic_type": "nested", "type": "object", "additionalProperties": { "anyOf": [ { "anyOf": [ { "type": "string", "format": "binary" }, { "type": "string" } ] }, { "type": "object" } ] } }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "id": { "title": "Id", "default": "", "type": "string" }, "last_updated": { "title": "Last Updated", "type": "string", "format": "date-time" }, "name": { "title": "Name", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "original": { "title": "Original", "default": "", "elastic_type": "text", "type": "string" }, "parsed": { "title": "Parsed", "default": "", "elastic_type": "text", "type": "string" } }, "additionalProperties": false, "definitions": { "Hash": { "title": "Hash", "description": "Hashes of raw data or a file.\n\n.. note::\n All hashes are uppercase hexadecimal strings, per :term:`ECS`", "type": "object", "properties": { "md5": { "title": "MD5 hash", "minLength": 32, "maxLength": 32, "type": "string" }, "sha1": { "title": "SHA1 hash", "minLength": 40, "maxLength": 40, "type": "string" }, "sha256": { "title": "SHA256 hash", "minLength": 64, "maxLength": 64, "type": "string" }, "sha512": { "title": "SHA512 hash", "minLength": 128, "maxLength": 128, "type": "string" } }, "additionalProperties": false }, "File": { "title": "File", "description": "Contextual information and metadata for a file.\n\nThe file could be on disk, in memory, a directory, or simply\nrepresent an artifact that's known to be on the device but\nPEAT doesn't have the ability to access.", "type": "object", "properties": { "created": { "title": "Created", "type": "string", "format": "date-time" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "device": { "title": "Device", "default": "", "type": "string" }, "directory": { "title": "Directory", "default": "", "type": "string" }, "extension": { "title": "Extension", "default": "", "type": "string" }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "local_path": { "title": "Local Path", "type": "string", "format": "path" }, "path": { "title": "Path", "type": "string", "format": "path" }, "peat_module": { "title": "Peat Module", "default": "", "type": "string" }, "gid": { "title": "Gid", "default": "", "type": "string" }, "group": { "title": "Group", "default": "", "type": "string" }, "mime_type": { "title": "Mime Type", "default": "", "type": "string" }, "mode": { "title": "Mode", "default": "", "type": "string" }, "mtime": { "title": "Mtime", "type": "string", "format": "date-time" }, "name": { "title": "Name", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "type": "string", "format": "binary" }, "owner": { "title": "Owner", "default": "", "type": "string" }, "size": { "title": "Size", "minimum": 0, "type": "integer" }, "target_path": { "title": "Target Path", "type": "string", "format": "path" }, "type": { "title": "Type", "default": "", "enum": [ "file", "dir", "symlink", "" ], "type": "string" }, "uid": { "title": "Uid", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false } } }
- Fields:
- Validators:
strip_quotes»authorstrip_quotes»descriptionstrip_quotes»idstrip_quotes»namestrip_quotes»parsed
-
author:
ConstrainedStrValue¶ Name of the person/organization/program that wrote the logic.
- Validated by:
strip_quotes
-
created:
datetime|None¶ UTC timestamp of when the logic was first created (when the source project file was created) or fist uploaded to the device.
-
formats:
dict[str,Union[AnyStr,dict]]¶ Sub-formats the logic has been parsed into, such as
"structured_text"or"tc6". Device dependent.
-
hash:
Hash¶ Hashes of the raw unparsed logic (the contents of``original``).
Note
This may differ from the file hash, if present.
-
id:
ConstrainedStrValue¶ Project ID or a similar identifier for the logic, e.g. a machine-generated UUID for the logic stored by the device.
- Validated by:
strip_quotes
-
name:
ConstrainedStrValue¶ Project name or other such identifier for the logic, e.g. a human-readable name for the logic stored by the device.
- Validated by:
strip_quotes
-
original:
str¶ Unparsed device program logic, in whatever format makes sense for that device. It is the file(s) that make up the process state logic, e.g., the ladder logic on a PLC or the configured protection schemes on a substation relay.
- CertEntity[source]¶
Issuer or Subject in a x509 certificate.
ECS documentation: x509 Certificate Fields
Show JSON schema
{ "title": "CertEntity", "description": "Issuer or Subject in a x509 certificate.\n\n`ECS documentation: x509 Certificate Fields <https://www.elastic.co/guide/en/ecs/current/ecs-x509.html>`__", "type": "object", "properties": { "common_name": { "title": "Common Name", "default": "", "type": "string" }, "country": { "title": "Country", "default": "", "type": "string" }, "distinguished_name": { "title": "Distinguished Name", "default": "", "type": "string" }, "locality": { "title": "Locality", "default": "", "type": "string" }, "organization": { "title": "Organization", "default": "", "type": "string" }, "organizational_unit": { "title": "Organizational Unit", "default": "", "type": "string" }, "state_or_province": { "title": "State Or Province", "default": "", "type": "string" } }, "additionalProperties": false }
- Fields:
-
common_name:
ConstrainedStrValue¶ Common name (CN).
-
country:
ConstrainedStrValue¶ Country code.
-
distinguished_name:
ConstrainedStrValue¶ Distinguished Name (DN).
-
locality:
ConstrainedStrValue¶ Locality (L).
-
organization:
ConstrainedStrValue¶ Organization (O).
-
organizational_unit:
ConstrainedStrValue¶ Organizational Unit (OU).
-
state_or_province:
ConstrainedStrValue¶ State or province names (ST, S, or P).
- X509[source]¶
x509 certificate.
ECS documentation: x509 Certificate Fields
Show JSON schema
{ "title": "X509", "description": "x509 certificate.\n\n`ECS documentation: x509 Certificate Fields <https://www.elastic.co/guide/en/ecs/current/ecs-x509.html>`__", "type": "object", "properties": { "alternative_names": { "title": "Alternative Names", "default": [], "type": "array", "items": { "type": "string" } }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "issuer": { "title": "Issuer", "default": { "common_name": "", "country": "", "distinguished_name": "", "locality": "", "organization": "", "organizational_unit": "", "state_or_province": "" }, "allOf": [ { "$ref": "#/definitions/CertEntity" } ] }, "not_after": { "title": "Not After", "type": "string", "format": "date-time" }, "not_before": { "title": "Not Before", "type": "string", "format": "date-time" }, "original": { "title": "Original", "default": "", "type": "string" }, "public_key_algorithm": { "title": "Public Key Algorithm", "default": "", "type": "string" }, "public_key_curve": { "title": "Public Key Curve", "default": "", "type": "string" }, "public_key_exponent": { "title": "Public Key Exponent", "minimum": 0, "type": "integer" }, "public_key_size": { "title": "Public Key Size", "minimum": 0, "type": "integer" }, "serial_number": { "title": "Serial Number", "default": "", "type": "string" }, "signature_algorithm": { "title": "Signature Algorithm", "default": "", "type": "string" }, "subject": { "title": "Subject", "default": { "common_name": "", "country": "", "distinguished_name": "", "locality": "", "organization": "", "organizational_unit": "", "state_or_province": "" }, "allOf": [ { "$ref": "#/definitions/CertEntity" } ] }, "version_number": { "title": "Version Number", "default": "", "type": "string" } }, "additionalProperties": false, "definitions": { "Hash": { "title": "Hash", "description": "Hashes of raw data or a file.\n\n.. note::\n All hashes are uppercase hexadecimal strings, per :term:`ECS`", "type": "object", "properties": { "md5": { "title": "MD5 hash", "minLength": 32, "maxLength": 32, "type": "string" }, "sha1": { "title": "SHA1 hash", "minLength": 40, "maxLength": 40, "type": "string" }, "sha256": { "title": "SHA256 hash", "minLength": 64, "maxLength": 64, "type": "string" }, "sha512": { "title": "SHA512 hash", "minLength": 128, "maxLength": 128, "type": "string" } }, "additionalProperties": false }, "CertEntity": { "title": "CertEntity", "description": "Issuer or Subject in a x509 certificate.\n\n`ECS documentation: x509 Certificate Fields <https://www.elastic.co/guide/en/ecs/current/ecs-x509.html>`__", "type": "object", "properties": { "common_name": { "title": "Common Name", "default": "", "type": "string" }, "country": { "title": "Country", "default": "", "type": "string" }, "distinguished_name": { "title": "Distinguished Name", "default": "", "type": "string" }, "locality": { "title": "Locality", "default": "", "type": "string" }, "organization": { "title": "Organization", "default": "", "type": "string" }, "organizational_unit": { "title": "Organizational Unit", "default": "", "type": "string" }, "state_or_province": { "title": "State Or Province", "default": "", "type": "string" } }, "additionalProperties": false } } }
- Fields:
-
issuer:
CertEntity¶ Issuing certificate authority.
-
public_key_algorithm:
ConstrainedStrValue¶ Algorithm used to generate the public key.
-
public_key_curve:
ConstrainedStrValue¶ The curve used by the elliptic curve public key algorithm.
-
public_key_exponent:
ConstrainedIntValue|None¶ Exponent used to derive the public key.
- Constraints:
minimum = 0
-
public_key_size:
ConstrainedIntValue|None¶ The size of the public key space in bits.
- Constraints:
minimum = 0
-
serial_number:
ConstrainedStrValue¶ Unique serial number issued by the certificate authority.
For consistency, if this value is alphanumeric, it should be formatted without colons and uppercase characters.
-
signature_algorithm:
ConstrainedStrValue¶ Identifier for certificate signature algorithm.
-
subject:
CertEntity¶ Certificate subject.
-
version_number:
ConstrainedStrValue¶ Version of x509 format.
- UEFIHash[source]¶
UEFI model that specifically labels objects from a UEFI file hash file. This model is different because it includes all file systems, not just the EFI File system
File system is either FS0 or FS1 pathname is the pathname of the files in the file system hash is the SHA256 hash of the files computed via python script
Show JSON schema
{ "title": "UEFIHash", "description": "UEFI model that specifically labels objects from a UEFI file hash file.\nThis model is different because it includes all file systems, not just\nthe EFI File system\n\nFile system is either FS0 or FS1\npathname is the pathname of the files in the file system\nhash is the SHA256 hash of the files computed via python script", "type": "object", "properties": { "file_system": { "title": "File System", "default": "", "type": "string" }, "pathname": { "title": "Pathname", "default": "", "type": "string" }, "hash": { "title": "Hash", "default": "", "type": "string" } }, "additionalProperties": false }
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- UEFIFile[source]¶
UEFI model that specifically labels objects from a UEFIExtract report file. This model is different because it includes only the SPI file system included in an SPI file dump type is the type of entry. Examples are “Region, Volume” subtype is the subtype of the entry. Can be blank, can be empty or invalid base is the start of location in memory in HEX Size is the end of location in memory in HEX CRC32 is the calculate crc32 for the file Name is the name of the file path is the path of the file since the dumps are given in a file like structure
Show JSON schema
{ "title": "UEFIFile", "description": "UEFI model that specifically labels objects from a UEFIExtract report file.\nThis model is different because it includes only the SPI file system\nincluded in an SPI file dump\ntype is the type of entry. Examples are \"Region, Volume\"\nsubtype is the subtype of the entry. Can be blank, can be empty or invalid\nbase is the start of location in memory in HEX\nSize is the end of location in memory in HEX\nCRC32 is the calculate crc32 for the file\nName is the name of the file\npath is the path of the file since the dumps are given in a file like\nstructure", "type": "object", "properties": { "type": { "title": "Type", "default": "", "type": "string" }, "subtype": { "title": "Subtype", "default": "", "type": "string" }, "base": { "title": "Base", "default": "", "type": "string" }, "size": { "title": "Size", "default": "", "type": "string" }, "crc32": { "title": "Crc32", "default": "", "type": "string" }, "guid": { "title": "Guid", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "path": { "title": "Path", "default": "", "type": "string" }, "created": { "title": "Created", "type": "string", "format": "date-time" } }, "additionalProperties": false }
- Fields:
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- Service[source]¶
Communication protocol “service” configured or running on the device.
Services can be over a variety of transports, including IP/Ethernet, serial direct, cellular, serial bus, field bus, etc.
Show JSON schema
{ "title": "Service", "description": "Communication protocol \"service\" configured or running on the device.\n\nServices can be over a variety of transports, including IP/Ethernet,\nserial direct, cellular, serial bus, field bus, etc.", "type": "object", "properties": { "configured_port": { "title": "Configured Port", "minimum": 0, "maximum": 65535, "type": "integer" }, "enabled": { "title": "Enabled", "type": "boolean" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "listen_address": { "title": "Listen Address", "default": "", "elastic_type": "ip", "type": "string" }, "listen_interface": { "title": "Listen Interface", "default": "", "type": "string" }, "process_name": { "title": "Process Name", "default": "", "type": "string" }, "process_pid": { "title": "Process Pid", "minimum": 0, "type": "integer" }, "port": { "title": "Port", "minimum": 1, "maximum": 65535, "type": "integer" }, "protocol": { "title": "Protocol", "default": "", "type": "string" }, "protocol_id": { "title": "Protocol Id", "default": "", "type": "string" }, "role": { "title": "Role", "default": "", "type": "string" }, "status": { "title": "Status", "default": "", "enum": [ "open", "closed", "verified", "" ], "type": "string" }, "transport": { "title": "Transport", "default": "", "type": "string" } }, "additionalProperties": false }
- Fields:
- Validators:
clean_protocol»protocolvalidate_ip»listen_address
-
configured_port:
ConstrainedIntValue|None¶ Port the service is configured to listen on (for TCP or UDP transports).
This field should only be set from values read from a device configuration, e.g. a config file, config dump, project file, etc. It should NOT be set using information from a live port list, scanning, etc.
This is intended to supplement the “port” field, e.g. if the listening port differs from what’s in the config, that’s forensically interesting.
Note
The value must be between 0 and 65,535. Port 0 is allowed for the
configured_portfield, but not theportfield, since there may be cases when it’s set to 0 in a config (e.g. to disable).Examples
80161502
- Constraints:
minimum = 0
maximum = 65535
-
enabled:
bool|None¶ If the service is enabled in the device configuration.
Warning
This can differ from
status, don’t assume they will match!
-
extra:
dict¶ Additional unstructured information about the service, such as a banner grab, odd behavior, or other miscellaneous data.
-
process_pid:
ConstrainedIntValue|None¶ Process ID associated with the service. This is the PID of the network service’s process.
- Constraints:
minimum = 0
-
port:
ConstrainedIntValue|None¶ Port the service is listening on (for TCP or UDP transports).
Note
The value must be between 1 and 65,535. While a port of
0is technically accurate, it’s not allowed since it’s not something that should be seen in the real world, and if it is, then there’s probably a bug in PEAT or one of it’s modules.Examples
80161502
- Constraints:
minimum = 1
maximum = 65535
-
protocol:
ConstrainedStrValue¶ Protocol name of the service. Must be lowercase with underscore separators. Format will be automatically checked and enforced. This is a short name or acronym, not an expanded or colloquial name.
Examples
httpsnmpmodbus_tcpicmp
- Validated by:
clean_protocol
-
protocol_id:
ConstrainedStrValue¶ Unique protocol identifier for the device, such as the Modbus Unit ID.
Examples
"10""119"
-
role:
ConstrainedStrValue¶ The operational role of the device for a given protocol.
-
status:
Literal['open','closed','verified','']¶ State of the service.
Note
verifiedmeans verified over the a live connection, not just read from a configuration file. Instead, theenabledfield should be used to reflect the configuration state.Valid values
open: something is listening, though it may not be the named serviceclosed: port is not able to be accessed.verified: service was positively identified (high certainty)"": the live status is unknown, such as when seen ina configuration or project file parsed offline.
- Interface[source]¶
Communication interface, such as a Ethernet port or Serial link.
Note
Currently, the
ip,subnet_mask, andgatewayfields are assumed to be IP version 4 (IPv4). However, they can and will hold IPv6 values in the future when PEAT adds IPv6 support.Show JSON schema
{ "title": "Interface", "description": "Communication interface, such as a Ethernet port or Serial link.\n\n.. note::\n Currently, the ``ip``, ``subnet_mask``, and ``gateway`` fields are\n assumed to be :term:`IP` version 4 (IPv4). However, they can and\n will hold IPv6 values in the future when PEAT adds IPv6 support.", "type": "object", "properties": { "alias": { "title": "Alias", "default": "", "type": "string" }, "application": { "title": "Application", "default": "", "type": "string" }, "connected": { "title": "Connected", "type": "boolean" }, "description": { "title": "Description", "default": { "brand": "", "contact_info": "", "description": "", "full": "", "model": "", "product": "", "vendor": { "id": "", "name": "" } }, "allOf": [ { "$ref": "#/definitions/Description" } ] }, "duplex": { "title": "Duplex", "default": "", "enum": [ "half", "full", "auto", "" ], "type": "string" }, "enabled": { "title": "Enabled", "type": "boolean" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "name": { "title": "Name", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "type": "string" }, "hostname": { "title": "Hostname", "default": "", "type": "string" }, "mac": { "title": "Mac", "default": "", "type": "string" }, "mac_vendor": { "title": "Mac Vendor", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "mtu": { "title": "Mtu", "elastic_type": "integer", "exclusiveMinimum": 0, "type": "integer" }, "physical": { "title": "Physical", "type": "boolean" }, "promiscuous_mode": { "title": "Promiscuous Mode", "type": "boolean" }, "speed": { "title": "Speed", "elastic_type": "integer", "minimum": 0, "type": "integer" }, "uptime": { "title": "Uptime", "type": "number", "format": "time-delta" }, "hardware_mac": { "title": "Hardware Mac", "default": "", "type": "string" }, "id": { "title": "Id", "default": "", "type": "string" }, "ip": { "title": "Ip", "default": "", "elastic_type": "ip", "type": "string" }, "subnet_mask": { "title": "Subnet Mask", "default": "", "elastic_type": "ip", "type": "string" }, "gateway": { "title": "Gateway", "default": "", "elastic_type": "ip", "type": "string" }, "serial_port": { "title": "Serial Port", "default": "", "type": "string" }, "baudrate": { "title": "Baudrate", "elastic_type": "integer", "exclusiveMinimum": 0, "type": "integer" }, "data_bits": { "title": "Data Bits", "elastic_type": "byte", "minimum": 0, "type": "integer" }, "parity": { "title": "Parity", "default": "", "enum": [ "none", "even", "odd", "" ], "type": "string" }, "stop_bits": { "title": "Stop Bits", "elastic_type": "byte", "minimum": 0, "type": "integer" }, "flow_control": { "title": "Flow Control", "default": "", "type": "string" }, "services": { "title": "Services", "default": [], "type": "array", "items": { "$ref": "#/definitions/Service" } }, "version": { "title": "Version", "default": "", "type": "string" } }, "additionalProperties": false, "definitions": { "Vendor": { "title": "Vendor", "description": "Identifies a device vendor (SEL, Schneider Electric, Siemens, etc).", "type": "object", "properties": { "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" } }, "additionalProperties": false }, "Description": { "title": "Description", "description": "Identifying information such as vendor, brand, and model.", "type": "object", "properties": { "brand": { "title": "Brand", "default": "", "type": "string" }, "contact_info": { "title": "Contact Info", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "full": { "title": "Full", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "model": { "title": "Model", "default": "", "type": "string" }, "product": { "title": "Product", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "vendor": { "title": "Vendor", "default": { "id": "", "name": "" }, "allOf": [ { "$ref": "#/definitions/Vendor" } ] } }, "additionalProperties": false }, "Service": { "title": "Service", "description": "Communication protocol \"service\" configured or running on the device.\n\nServices can be over a variety of transports, including IP/Ethernet,\nserial direct, cellular, serial bus, field bus, etc.", "type": "object", "properties": { "configured_port": { "title": "Configured Port", "minimum": 0, "maximum": 65535, "type": "integer" }, "enabled": { "title": "Enabled", "type": "boolean" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "listen_address": { "title": "Listen Address", "default": "", "elastic_type": "ip", "type": "string" }, "listen_interface": { "title": "Listen Interface", "default": "", "type": "string" }, "process_name": { "title": "Process Name", "default": "", "type": "string" }, "process_pid": { "title": "Process Pid", "minimum": 0, "type": "integer" }, "port": { "title": "Port", "minimum": 1, "maximum": 65535, "type": "integer" }, "protocol": { "title": "Protocol", "default": "", "type": "string" }, "protocol_id": { "title": "Protocol Id", "default": "", "type": "string" }, "role": { "title": "Role", "default": "", "type": "string" }, "status": { "title": "Status", "default": "", "enum": [ "open", "closed", "verified", "" ], "type": "string" }, "transport": { "title": "Transport", "default": "", "type": "string" } }, "additionalProperties": false } } }
- Fields:
- Validators:
cleanstr»applicationcleanstr»flow_controlcleanstr»paritycleanstr»typestrip_quotes»descriptionvalidate_ip»gatewayvalidate_ip»ipvalidate_ip»subnet_maskvalidate_mac»hardware_macvalidate_mac»mac
-
alias:
str¶ Interface alias as reported by the system, typically used in firewall implementations for e.g. inside, outside, or dmz logical interface naming.
-
application:
str¶ Higher-level communication protocol being used regardless of whether the device is connected via serial or IP.
This field should be lowercase and without separators, when possible, or with underscore (
_) separators otherwise.Examples
modbusdnp3sel
- Validated by:
cleanstr
-
connected:
bool|None¶ If the interface is currently connected to something (e.g. carrier signal on Ethernet or connected to a tower for wireless interfaces).
-
description:
Description¶ Identifying information for the interface’s hardware or software, such as vendor, brand, and model.
- Validated by:
strip_quotes
-
duplex:
Literal['half','full','auto','']¶ Duplex mode for Ethernet interfaces.
Allowed values
half
full
auto
“” (empty string)
-
extra:
dict¶ Additional unstructured information related to the interface, generally this is vendor-specific information.
-
name:
ConstrainedStrValue¶ Interface name, as defined by the device.
For example, SEL relays refer to the serial and network ports by names such as
1,2,3, orF.Examples
PFens0eth1F
-
type:
str¶ The type of physical communication medium the communication interface utilizes. Lowercase, underscore separators.
Examples
ethernetloopbackpoint_to_pointrs_232rs_422rs_485usb
- Validated by:
cleanstr
-
hostname:
ConstrainedStrValue¶ Hostname of the network interface.
Examples
some-relay.local
-
mac:
str¶ The IEEE 802 standard 48-bit MAC address of the interface. This is the current MAC address used by the interface.
Only applicable to Ethernet-type interfaces. The MAC address is formatted as a uppercase colon-separated string.
Examples
00:00:00:FF:FF:FF
- Validated by:
validate_mac
-
mac_vendor:
str¶ Vendor name resolved from the MAC address OUI.
This field is auto-populated by PEAT if the
macfield is set.
-
mtu:
PositiveInt|None¶ Maximum Transmission Unit (MTU) size configured for the interface. This generally only applies to Ethernet interfaces.
- Constraints:
exclusiveMinimum = 0
-
physical:
bool|None¶ If the interface is a physical interface (e.g. is a port on the device). If false, then it’s likely a virtual interface or software-defined. Use the “type” and “description” fields to store additional details.
-
speed:
ConstrainedIntValue|None¶ Transmission rate of the interface, in Mbps (megabits per second). Example: for Gigabit Ethernet, this would be 1000.
- Constraints:
minimum = 0
-
uptime:
timedelta|None¶ How long the interface has been connected, in milliseconds or as a
timedeltainstance.NOTE: normal integers can be assigned to this! (e.g.
iface.uptime = 123)
-
hardware_mac:
str¶ The hardware MAC address of the interface. This is intrinsic to the physical NIC, and may differ from the MAC address currently in use by the interface.
Only applicable to Ethernet-type interfaces. The MAC address is formatted as a uppercase colon-separated string.
Examples
00:00:00:FF:FF:FF
- Validated by:
validate_mac
-
id:
ConstrainedStrValue¶ Identifier for the interface. The meaning of this value is device-dependent.
-
ip:
str¶ The IP address of the interface. This is usually applicable to Ethernet-type interfaces, but could be applicable to Serial interfaces as well (e.g. on SEL devices).
Examples
192.0.2.123
- Validated by:
validate_ip
-
subnet_mask:
str¶ IP subnet mask of the interface.
Examples
255.255.255.0255.255.255.192
- Validated by:
validate_ip
-
gateway:
str¶ IPv4 address of the default gateway of the interface.
Examples
192.0.2.1
- Validated by:
validate_ip
-
serial_port:
ConstrainedStrValue¶ Serial port on the local system connected to the device. This could be a Windows COM port, e.g.
COM4, or a Linux file path, e.g./dev/ttyS0. This is also used for USB connections.Examples
COM4/dev/ttyS0/dev/ttyUSB0
-
baudrate:
PositiveInt|None¶ Data rate for a serial link.
Examples
56700
- Constraints:
exclusiveMinimum = 0
-
data_bits:
ConstrainedIntValue|None¶ Number of data bits for a serial link.
Examples
8
- Constraints:
minimum = 0
-
parity:
Literal['none','even','odd','']¶ Parity setting for a serial link.
Allowed values
none
even
odd
“” (empty string)
- Validated by:
cleanstr
-
stop_bits:
ConstrainedIntValue|None¶ Number of stop bits for a serial link.
Examples
0
1
- Constraints:
minimum = 0
- Register[source]¶
Configured I/O protocol data point (“registers”), e.g. DNP3 or Modbus/TCP.
Show JSON schema
{ "title": "Register", "description": "Configured I/O protocol data point (\"registers\"), e.g. DNP3 or Modbus/TCP.", "type": "object", "properties": { "address": { "title": "Address", "default": "", "type": "string" }, "data_type": { "title": "Data Type", "default": "", "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": "text", "type": "string" }, "enabled": { "title": "Enabled", "type": "boolean" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "group": { "title": "Group", "default": "", "type": "string" }, "io": { "title": "Io", "default": "", "type": "string" }, "measurement_type": { "title": "Measurement Type", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "protocol": { "title": "Protocol", "default": "", "type": "string" }, "read_write": { "title": "Read Write", "default": "", "enum": [ "read", "write", "read_write", "" ], "type": "string" }, "tag": { "title": "Tag", "default": "", "type": "string" } }, "additionalProperties": false }
- Fields:
- Validators:
clean_protocol»protocolcleanstr»data_typecleanstr»protocolstrip_quotes»description
-
address:
ConstrainedStrValue¶ Address of the data. Tells protocol parser how to identify a data field in a packet. A number, string, or more complex identifier. For Object Oriented protocols, this field flattens the data_address.
Examples
12123456pump-jack-six-exampledevice-example_1234_trend-log
-
data_type:
str¶ Data type of the register. Tells the user or code reading our data how to interpret the field. Format: Lowercase, underscore-separated string.
Examples
float_16stringint_32
- Validated by:
cleanstr
-
description:
str¶ Human-readable description of the register (some device configurations or project files have this).
Examples
"Intake Fuel - Valve 1 - Second Boiler"
- Validated by:
strip_quotes
-
enabled:
bool|None¶ If the register is considored to be “enabled”, e.g. has a valid configuration or is otherwise enabled for use on the device.
-
group:
ConstrainedStrValue¶ Logical mapping or settings group (e.g. on SEL relays) associated with the Register.
Examples:
D1
D3
M
DNPA
-
io:
ConstrainedStrValue¶ I/O point it’s attached to (e.g. protocol register or physical I/O).
This allows direct reference to an IO object without requiring a Tag.
Examples
rtu-8_I0
-
measurement_type:
ConstrainedStrValue¶ Type of information the register is tracking (e.g analog I/O, Discrete I/O). Tells analytic which algorithms to deploy. For example, in Modbus a 16-bit register can track an event count (Discrete), a temperature (analog), or could be a set of 16 Boolean flags (alarms).
Examples
analogbinary
-
name:
ConstrainedStrValue¶ Name or unique descriptor of the register (if different from the address).
Examples
AI_99MOD_005
-
protocol:
str¶ The Parser uses this to distinguish protocols. Not all vendors follow the protocol spec. To indicate if this is a vendor- specific deviation from the standard, use the syntax
[protocol]_[device or vendor name].Examples
dnp3modbus
- Validated by:
clean_protocolcleanstr
-
read_write:
Literal['read','write','read_write','']¶ Direction of information flow. Is register read, write, or both?
Allowed values
readwriteread_write
-
tag:
ConstrainedStrValue¶ Register tag given in config file. Provides analytic with some register context. May be a human-readable display name.
Examples
valve_1
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- Tag[source]¶
Variable in a ICS/OT device, often mapping to physical I/O.
These are commonly stored in a “tag database” in a SCADA system or the configuration of a device.
Show JSON schema
{ "title": "Tag", "description": "Variable in a :term:`ICS`/:term:`OT` device, often mapping to physical I/O.\n\nThese are commonly stored in a \"tag database\" in a :term:`SCADA` system or\nthe configuration of a device.", "type": "object", "properties": { "address": { "title": "Address", "default": "", "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": "text", "type": "string" }, "io": { "title": "Io", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "type": "string" } }, "additionalProperties": false }
- Fields:
- Validators:
cleanstr»typestrip_quotes»description
-
address:
ConstrainedStrValue¶ Address of the tag.
Examples
29
-
io:
ConstrainedStrValue¶ I/O point it’s attached to (e.g. protocol register or physical I/O).
Examples
rtu-8_I0
-
name:
ConstrainedStrValue¶ Tag name or label (e.g. how it’s referenced).
Examples
var_rtu-8_I0
-
type:
str¶ Data type of the tag, lowercase and underscore-separated.
Examples
analogbinary
- Validated by:
cleanstr
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- IO[source]¶
Physical Input/Output (I/O) connections on a device.
Physical I/O points are distinct from
Register, which handle communication protocols and may not necessarily map to physical I/O.Physical I/O points are typically referenced by a
Tag, though this may not always be the case.On module-based devices like a PLC, Physical I/O points may be associated with a module, however this may not always be the case.
Show JSON schema
{ "title": "IO", "description": "Physical Input/Output (I/O) connections on a device.\n\nPhysical I/O points are distinct from :class:`~peat.data.models.Register`,\nwhich handle communication protocols and may not necessarily map to physical I/O.\n\nPhysical I/O points are typically referenced by a :class:`~peat.data.models.Tag`,\nthough this may not always be the case.\n\nOn module-based devices like a :term:`PLC`, Physical I/O points may be associated\nwith a module, however this may not always be the case.", "type": "object", "properties": { "address": { "title": "Address", "default": "", "type": "string" }, "description": { "title": "Description", "default": "", "elastic_type": "text", "type": "string" }, "direction": { "title": "Direction", "default": "", "enum": [ "input", "output", "" ], "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "type": "string" }, "slot": { "title": "Slot", "default": [], "type": "array", "items": { "type": "string" } } }, "additionalProperties": false }
- Fields:
- Validators:
cleanstr»directionstrip_quotes»description
-
address:
ConstrainedStrValue¶ Address of the I/O point (if applicable).
Examples
29
-
direction:
Literal['input','output','']¶ Direction of the I/O point.
Allowed values
inputoutput
- Validated by:
cleanstr
-
id:
ConstrainedStrValue¶ ID of the I/O point.
Examples
rtu-1_I16O0
-
name:
ConstrainedStrValue¶ I/O point name or label (typically referenced by a
Tag).Examples
var_rtu-1_I16
-
type:
ConstrainedStrValue¶ Data type of the I/O point. Possible values are device-dependent.
Examples
analogbinaryEBOOLDATE
-
slot:
list[ConstrainedStrValue]¶ Slot number(s) of the module(s) the point is associated with, if any.
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- LatLon[source]¶
Latitude and Longitude (geographical coordinates).
Show JSON schema
{ "title": "LatLon", "description": "Latitude and Longitude (geographical coordinates).", "type": "object", "properties": { "lat": { "title": "Latitude", "elastic_type": "double", "minimum": -90.0, "maximum": 90.0, "type": "number" }, "lon": { "title": "Longitude", "elastic_type": "double", "minimum": -180.0, "maximum": 180.0, "type": "number" } }, "additionalProperties": false }
- Fields:
- Geo[source]¶
Geolocation information (the device’s physical location).
Show JSON schema
{ "title": "Geo", "description": "Geolocation information (the device's physical location).", "type": "object", "properties": { "city_name": { "title": "City Name", "default": "", "type": "string" }, "country_name": { "title": "Country Name", "default": "", "type": "string" }, "location": { "title": "Location", "default": { "lat": null, "lon": null }, "elastic_type": "geo_point", "allOf": [ { "$ref": "#/definitions/LatLon" } ] }, "name": { "title": "Name", "default": "", "type": "string" }, "timezone": { "title": "Timezone", "default": "", "type": "string" } }, "additionalProperties": false, "definitions": { "LatLon": { "title": "LatLon", "description": "Latitude and Longitude (geographical coordinates).", "type": "object", "properties": { "lat": { "title": "Latitude", "elastic_type": "double", "minimum": -90.0, "maximum": 90.0, "type": "number" }, "lon": { "title": "Longitude", "elastic_type": "double", "minimum": -180.0, "maximum": 180.0, "type": "number" } }, "additionalProperties": false } } }
- Fields:
-
city_name:
ConstrainedStrValue¶ Name of the city where the device is physically located.
Examples
Albuquerque
-
country_name:
ConstrainedStrValue¶ Name of the country where the device is physically located, in whatever form is reasonable.
Examples
USAUnited States of AmericaCanada
-
name:
ConstrainedStrValue¶ Custom location name, as retrieved from the device.
Examples
abq-dc1st floor network closet
-
timezone:
ConstrainedStrValue¶ Timezone configured for the device.
Acceptable timezone formats are: a canonical ID (e.g.
America/Denver) or abbreviated (e.g.EST). Canonical ID is preferred for PEAT.Examples
America/DenverEtc/UTCESTMSTUTC
- Event[source]¶
Device log entry, such as logins, metering reads, or system events.
Show JSON schema
{ "title": "Event", "description": "Device log entry, such as logins, metering reads, or system events.", "type": "object", "properties": { "action": { "title": "Action", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "category": { "title": "Category", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "created": { "title": "Created", "type": "string", "format": "date-time" }, "dataset": { "title": "Dataset", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "id": { "title": "Id", "default": "", "type": "string" }, "ingested": { "title": "Ingested", "type": "string", "format": "date-time" }, "kind": { "title": "Kind", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true }, "message": { "title": "Message", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "module": { "title": "Module", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "elastic_type": "text", "type": "string" }, "outcome": { "title": "Outcome", "default": "", "type": "string" }, "provider": { "title": "Provider", "default": "", "type": "string" }, "sequence": { "title": "Sequence", "type": "integer" }, "severity": { "title": "Severity", "default": "", "type": "string" }, "timezone": { "title": "Timezone", "default": "", "type": "string" }, "type": { "title": "Type", "default": [], "type": "array", "items": { "type": "string" }, "uniqueItems": true } }, "additionalProperties": false, "definitions": { "Hash": { "title": "Hash", "description": "Hashes of raw data or a file.\n\n.. note::\n All hashes are uppercase hexadecimal strings, per :term:`ECS`", "type": "object", "properties": { "md5": { "title": "MD5 hash", "minLength": 32, "maxLength": 32, "type": "string" }, "sha1": { "title": "SHA1 hash", "minLength": 40, "maxLength": 40, "type": "string" }, "sha256": { "title": "SHA256 hash", "minLength": 64, "maxLength": 64, "type": "string" }, "sha512": { "title": "SHA512 hash", "minLength": 128, "maxLength": 128, "type": "string" } }, "additionalProperties": false } } }
- Fields:
- Validators:
-
action:
ConstrainedStrValue¶ Type of event.
Examples
alarm
-
category:
set[str]¶ ECS category of the event, out of the allowed values defined by ECS. This is a set of values, and is an array in Elasticsearch, which allows for Kibana queries such as
host.event.type:alert and host.event.category:authentication.Allowed values
authenticationconfigurationdatabasedriverfilehostiamintrusion_detectionmalwarenetworkpackageprocessregistrysessionweb
- Validated by:
validate_ecs
-
dataset:
ConstrainedStrValue¶ What log the event came from. This is especially important on devices with multiple log types.
Examples
metering_reads
-
extra:
dict¶ Other event metadata that doesn’t fit anywhere in the model, but is still worth capturing.
-
ingested:
datetime|None¶ When the event was generated by PEAT, e.g. when it was parsed or pulled from a device.
Warning
This should almost always differ from
createdand the two should NOT be confused.
-
kind:
set[str]¶ Gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. Further reading This is a set of values, and is an array in Elasticsearch, which allows for Kibana queries such as
host.event.kind:event and host.event.type:deleted.Allowed values
alerteventmetricstatepipeline_error: Used for indicating there was an error processing the event
- Validated by:
validate_ecs
-
message:
ConstrainedStrValue¶ Simplified message body, for example a human-readable portion of the raw event. This should be set in addition to setting the
originalfield.
-
module:
ConstrainedStrValue¶ Name of the module this data is coming from, e.g. the PEAT module.
-
outcome:
ConstrainedStrValue¶ Outcome of the event. Further reading
Allowed values
successfailureunknown
- Validated by:
validate_ecs
-
provider:
ConstrainedStrValue¶ Source of the event. This is almost always the Device ID.
-
sequence:
int|None¶ Sequence number of the event. The sequence number is a value published by some event sources, to make the exact ordering of events unambiguous, regardless of the timestamp precision.
-
severity:
ConstrainedStrValue¶ Severity or log level of the event as stored on the device.
Examples
debugERR
-
timezone:
ConstrainedStrValue¶ Timezone for the event.
This field should be populated when the event’s timestamp does not include timezone information already. It’s optional otherwise.
Note
This field will be auto-populated from the device’s timezone field (DeviceData.geo.timezone), if the timestamp isn’t timezone-aware and the device’s timezone is known.
Acceptable timezone formats are: a canonical ID (e.g.
Europe/Amsterdam) or abbreviated (e.g.EST). Canonical ID is preferred for PEAT.Examples
Europe/AmsterdamAmerica/DenverEtc/UTCESTMSTUTC
-
type:
set[str]¶ List of event category “sub-buckets” the event falls under. The valid values depend on the value for
category, refer to the ECS documentation for details. This is a set of values, and is an array in Elasticsearch, which allows for Kibana queries such ashost.event.type:user and host.event.type:deleted.Allowed values (refer to the ECS documentation)
accessadminallowedchangeconnectioncreationdeletiondeniedenderrorgroupinfoinstallationprotocolstartuser
- Validated by:
validate_ecs
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- OS[source]¶
Operating System (OS) information, such as the name and version.
Show JSON schema
{ "title": "OS", "description": "Operating System (OS) information, such as the name and version.", "type": "object", "properties": { "family": { "title": "Family", "default": "", "type": "string" }, "full": { "title": "Full", "default": "", "type": "string" }, "kernel": { "title": "Kernel", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" }, "timestamp": { "title": "Timestamp", "type": "string", "format": "date-time" }, "vendor": { "title": "Vendor", "default": { "id": "", "name": "" }, "allOf": [ { "$ref": "#/definitions/Vendor" } ] }, "version": { "title": "Version", "default": "", "type": "string" } }, "additionalProperties": false, "definitions": { "Vendor": { "title": "Vendor", "description": "Identifies a device vendor (SEL, Schneider Electric, Siemens, etc).", "type": "object", "properties": { "id": { "title": "Id", "default": "", "type": "string" }, "name": { "title": "Name", "default": "", "type": "string" } }, "additionalProperties": false } } }
- Fields:
-
family:
ConstrainedStrValue¶ Operating system family, such as Debian, Windows, etc. Lowercase value. This can be general (e.g.
linux) or specific (e.g.debian).Examples
debianwindowslinux
-
full:
ConstrainedStrValue¶ Full operating system name, including the version or code name.
Examples
WindRiver VxWorks 7
-
kernel:
ConstrainedStrValue¶ Operating system kernel version as a raw string.
Examples:
4.4.0-112-generic
-
name:
ConstrainedStrValue¶ Operating system name, without the version.
Examples
VxWorks,Linux
-
timestamp:
datetime|None¶ Timestamp of the OS, as extracted from the device or firmware. Device-dependent meaning. Often represents when the OS was compiled/built or released.
-
version:
ConstrainedStrValue¶ Operating system version as a raw string.
- Memory[source]¶
Physical memory values (e.g. RAM, EEPROM).
Show JSON schema
{ "title": "Memory", "description": "Physical memory values (e.g. :term:`RAM`, EEPROM).", "type": "object", "properties": { "address": { "title": "Address", "default": "", "type": "string" }, "created": { "title": "Created", "type": "string", "format": "date-time" }, "dataset": { "title": "Dataset", "default": "", "type": "string" }, "device": { "title": "Device", "default": "", "type": "string" }, "process": { "title": "Process", "default": "", "type": "string" }, "size": { "title": "Size", "type": "integer" }, "value": { "title": "Value", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false }
- Fields:
- Validators:
-
address:
str¶ Starting address of the read, as a hexadecimal string.
This should be zero-padded hex bytes, without a leading hex identifier, and uppercase characters.
Examples:
00000003D3ADB33F
- Validated by:
validate_hex
-
created:
datetime|None¶ When the read occurred. Represents when in time the memory address had the value.
-
dataset:
ConstrainedStrValue¶ Data source of the memory read, such as the memory region or log it was extracted from, if applicable.
Examples
watchdog_loginternal_memoryRAMEEPROM
-
device:
ConstrainedStrValue¶ Device that was the source of the read. This is almost always the device ID.
-
process:
ConstrainedStrValue¶ Name of the system process or task this memory read is associated with.
-
value:
str¶ The value read from memory, as a hexadecimal string.
Each hex pair (e.g.
3f) represents 1 byte.The length of this string should be twice the value of
size(size*2).This should be zero-padded hex bytes, without a leading hex identifier, and uppercase characters.
Examples:
00000003D3ADB33F
- Validated by:
validate_hex
- gen_elastic_content(dev=None)[source]¶
Generates the data structure that will be pushed to Elasticsearch.
This is overridden by child classes.
- Parameters:
dev (
DeviceData|None) -- DeviceData object to use as source for host data for the event- Return type:
- Returns:
Complete document structure that can be pushed to Elasticsearch
- SSHKey[source]¶
SSH keys (public or private).
Show JSON schema
{ "title": "SSHKey", "description": "SSH keys (public or private).", "type": "object", "properties": { "description": { "title": "Description", "default": "", "type": "string" }, "file": { "title": "File", "default": { "created": null, "description": "", "device": "", "directory": "", "extension": "", "hash": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "local_path": null, "path": null, "peat_module": "", "gid": "", "group": "", "mime_type": "", "mode": "", "mtime": null, "name": "", "original": "", "owner": "", "size": null, "target_path": null, "type": "", "uid": "", "extra": {} }, "allOf": [ { "$ref": "#/definitions/File" } ] }, "host": { "title": "Host", "default": "", "type": "string" }, "id": { "title": "Id", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "type": "string" }, "type": { "title": "Type", "default": "", "enum": [ "public", "" ], "type": "string" }, "user": { "title": "User", "default": "", "type": "string" } }, "additionalProperties": false, "definitions": { "Hash": { "title": "Hash", "description": "Hashes of raw data or a file.\n\n.. note::\n All hashes are uppercase hexadecimal strings, per :term:`ECS`", "type": "object", "properties": { "md5": { "title": "MD5 hash", "minLength": 32, "maxLength": 32, "type": "string" }, "sha1": { "title": "SHA1 hash", "minLength": 40, "maxLength": 40, "type": "string" }, "sha256": { "title": "SHA256 hash", "minLength": 64, "maxLength": 64, "type": "string" }, "sha512": { "title": "SHA512 hash", "minLength": 128, "maxLength": 128, "type": "string" } }, "additionalProperties": false }, "File": { "title": "File", "description": "Contextual information and metadata for a file.\n\nThe file could be on disk, in memory, a directory, or simply\nrepresent an artifact that's known to be on the device but\nPEAT doesn't have the ability to access.", "type": "object", "properties": { "created": { "title": "Created", "type": "string", "format": "date-time" }, "description": { "title": "Description", "default": "", "elastic_type": { "type": "keyword", "fields": { "text": { "type": "text" } } }, "type": "string" }, "device": { "title": "Device", "default": "", "type": "string" }, "directory": { "title": "Directory", "default": "", "type": "string" }, "extension": { "title": "Extension", "default": "", "type": "string" }, "hash": { "title": "Hash", "default": { "md5": null, "sha1": null, "sha256": null, "sha512": null }, "allOf": [ { "$ref": "#/definitions/Hash" } ] }, "local_path": { "title": "Local Path", "type": "string", "format": "path" }, "path": { "title": "Path", "type": "string", "format": "path" }, "peat_module": { "title": "Peat Module", "default": "", "type": "string" }, "gid": { "title": "Gid", "default": "", "type": "string" }, "group": { "title": "Group", "default": "", "type": "string" }, "mime_type": { "title": "Mime Type", "default": "", "type": "string" }, "mode": { "title": "Mode", "default": "", "type": "string" }, "mtime": { "title": "Mtime", "type": "string", "format": "date-time" }, "name": { "title": "Name", "default": "", "type": "string" }, "original": { "title": "Original", "default": "", "type": "string", "format": "binary" }, "owner": { "title": "Owner", "default": "", "type": "string" }, "size": { "title": "Size", "minimum": 0, "type": "integer" }, "target_path": { "title": "Target Path", "type": "string", "format": "path" }, "type": { "title": "Type", "default": "", "enum": [ "file", "dir", "symlink", "" ], "type": "string" }, "uid": { "title": "Uid", "default": "", "type": "string" }, "extra": { "title": "Extra", "default": {}, "elastic_type": "flattened", "type": "object" } }, "additionalProperties": false } } }
- Fields:
-
description:
ConstrainedStrValue¶ Description of the SSH key and/or any comments.
-
host:
ConstrainedStrValue¶ Host associated with the key (hostname, DNS name, or IP).
-
id:
ConstrainedStrValue¶ Unique identifier for the key, if any.
-
original:
ConstrainedStrValue¶ Complete contents of the key, with any trailing whitespace removed.
-
user:
ConstrainedStrValue¶ Name of user associated with the key.
- process_file(file)[source]¶
Transform a
dictinto aFileobject and populate unfilled fields.- Return type:
- process_file_extension(file)[source]¶
Automatically infer and populate unset fields on a
Fileobject.- Return type:
- annotate_obj_and_file(obj, field_name, dev)[source]¶
Populate original field if not set, and save data to file if it hasn’t been.
- Return type:
- export_models_to_elastic(models, dev, elastic=None)[source]¶
Export model objects to an Elasticsearch database.
Under the hood, this uses the Elasticsearch Bulk API to do efficient exporting in parallel and with fewer API requests.
- Parameters:
models (
list[BaseModel]) -- the models to export. All models in the list must be of the same type (don’t mix models). To export a single model, wrap the model in a list, e.g.models=[mymodel].dev (
DeviceData) -- the DeviceData object the model(s) are associated withelastic (
Elastic|None) -- TheElasticinstance to use. If unspecified, this defaults to the globalElasticinstance inelastic.
- Return type:
- Returns:
True if the export was successful, False if there were any errors