MQTT to InfluxDB

Description

ezvpn-mqtt-influxdb listens on an MQTT broker and writes data to an InfluxDB V1 database.

In IOhubTM architecture, ezvpn-mqtt-influxdb will subscribe fld/+/r/# topic to the ezvpn-mqtt broker, and store data read to an InfluxDB v1.

MQTT to InfluxDB Schema

How to use it

ezvpn-mqtt-influxdb is a Docker container image pre-configured for communicate with the ezvpn-mqtt and ezvpn-influxdb containers. To use it, you will need to install it along with ezvpn-mqtt and ezvpn-influxdb, and it will be ready to use.

Buffer

You can enable buffer using MQTT_WANT_QOS2 environment variable. Check retry on error variables to configure error handling.

Retention Policy

No retention policy is set by default. Data sent to the database uses the InfluxDB default retention policy.

You can change the retention policy of the data sent through this specific container setting INFLUXDB_RP_DURATION and INFLUXDB_RP_NAME.

If the policy in INFLUXDB_RP_NAME does not exist on Influx db, it it created. Otherwise it is updated. In both case, the new policy is defined by INFLUXDB_RP_DURATION.

If you redefine a retention policy, you are affecting all the other Influx clients using the same policy.

Tags

If you need to collect information from multiple sources to the same database, you might want to associate tags to classify each measure.

Using the TAGS variable, the data is sent to InfluxDB adding the tags to the measurement. TAGS must be a JSON object; each property

{
    "property1": "<value>",
    "property2": "<value>",
    "property3": "<value>"
}

If the TAGS variable is defined, it has to be a JSON object. Each property will be used as a tag whose name is the property; the property value (it must be a string) will be used as the tag value.

In the example below, the value of TAGS is

{
    "company": "ACME",
    "machine_name": "cutter 23",
    "serial_number": "1234567890"
}

and the recorded measure will have 3 tags (company, machine_name, serial_number) with values 'ACME', 'cutter 23', '1234567890'.

Each tag property must be named keeping into account the InfluxDB naming format, avoiding for example keywords like select, group by, etc.

Environment variables

When you start the ezvpn-mqtt-influxdb container, you can adjust the instance's configuration by passing one or more environment variables to the docker run command.

MQTT Connection

  • MQTT_HOST: MQTT host. Defaults to 127.0.0.1
  • MQTT_PORT: MQTT port. Defaults to 1883
  • MQTT_IN_TOPIC: IOhubTM MQTT topic to subscribe (e.g. fld/modbus/r/#). Defaults to fld/+/r/#. Subscribe to multiple topics can be obtained defining each topic on a separated line.
  • MQTT_WANT_QOS2: ask MQTT to upgrade each incoming message on MQTT_IN_TOPIC to QoS 2 (delivery guaranteed). MQTT will honor the request only if persistence is enabled.

InfluxDB Connection

  • INFLUXDB_HOST: InfluxDB host. Defaults to 127.0.0.1
  • INFLUXDB_PORT: InfluxDB port. Defaults to 8086
  • INFLUXDB_DB: InfluxDB database. Defaults to data
  • INFLUXDB_RP_DURATION: InfluxDB retention policy duration to apply. If not defined, the existing default retention policy is used. For example, 2w for two weeks, 7d for seven days; browse the InfluxDB docs to get a list of all the possible values.
  • INFLUXDB_RP_NAME: InfluxDB retention policy name. Defaults to the current default retention policy. Not used if INFLUXDB_RP_DURATION is empty.
  • INFLUXDB_RP_SET_DEFAULT: if true, the retention policy is set as the default one. Defaults to false. Not used if INFLUXDB_RP_DURATION is empty.

Retry on error

  • RETRY_ON_ERROR: if true, on InfluxDB communication failure, write is retried every RETRY_INTERVAL milliseconds, forever. Otherwise the message is discarded. Defaults to false.
  • RETRY_INTERVAL: interval in milliseconds between each write retry to InfluxDB. Defaults to 5000. Used only if RETRY_ON_ERROR is true.

Tags

TAGS: if defined, it is a JSON object containing additional tags (protocol is always sent as a tag) to send with each measure. Property names are used as tag names; property values are used as tag values. Property values must be strings.

Other variables

  • SKIP_RETAIN: if true, discard all incoming MQTT messages with retain flag set to true. Defaults to false.

Docker container details

Image: us-central1-docker.pkg.dev/ez-shared/iohub/iohub-mqtt-influxdb

Supported architecture: amd64

Changelog

v1.0.5
  • Warning on INFLUXDB_RP_DURATION present and INFLUXDB_RP_NAME missing
v1.0.4
  • Smaller image
v1.0.3
  • TAGS Environment variable added
v1.0.2
  • INFLUXDB_RP_DURATION Environment variable added
  • INFLUXDB_RP_NAME Environment variable added
  • INFLUXDB_RP_SET_DEFAULT Environment variable added
  • MQTT_IN_TOPIC accepts multiple topics
v1.0.1
  • MQTT_WANT_QOS2 Environment variable added
  • RETRY_ON_ERROR Environment variable added
  • RETRY_INTERVAL Environment variable added
  • SKIP_RETAIN Environment variable added
v1.0.0
  • First Release