MQTT to MQTT

Deprecated

This container is deprecated please use the new version of MQTT to MQTT instead.

Description

ezvpn-mqtt-mqtt subscribes to an MQTT broker and publishes all events to another MQTT server.

In IOhubTM architecture, ezvpn-mqtt-mqtt will subscribe to the # topic on a source MQTT broker and publish every incoming message to a destination MQTT broker.

You can use this container when needed to send data to a destination MQTT server.

Generic MQTT Schema

How to use it

ezvpn-mqtt-mqtt is a Docker container container based on Eclipse Mosquitto pre-configured for communication with ezvpn-mqtt and a generic MQTT broker. To use, install it along with ezvpn-mqtt and a destination MQTT broker. You can anyway point the source broker to any other address.

If you need bidirectional connection, you can install a second container instance, sending messages in the opposite direction.

Buffer

You can enable buffer using MQTT_WANT_QOS2 environment variable. Check retry on error variables to configure error handling.

Metadata

You can add metadata to the measurement using one of the following methods: Embedded and Detached.

Embedded Metadata

Using the METADATA variable, its content is added to each measurement, in the metadata property, as a string. Any string value can be used for the METADATA.

Detached Metadata

To minimize MQTT traffic, it is possible to avoid sending the metadata along with every message. The detached metadata will be sent, on the container startup, to a specific topic.

To enable the detached metadata, USE_METADATA_CHANNEL must be true.

If detached metadata is enabled, the following payload will be sent to the topic METADATA_TOPIC:

{
    "id": "<the value of METADATA_ID>",
    "metadata": "<the value of METADATA_FOR_CHANNEL>"
}

Each data payload sent will have the metadataId property (instead of the metadata property used in the embedded mode) containing the value of METADATA_ID.

Environment variables

When you start the ezvpn-mqtt-mqtt container, you can adjust the instance's configuration by passing one or more environment variables to the docker run command.

Source MQTT connection

  • MQTT_IN_HOST: MQTT source host. Defaults to 127.0.0.1
  • MQTT_IN_PORT: MQTT source port. Defaults to 1883
  • MQTT_IN_USE_URL_AUTH: If true MQTT_IN_USERNAME and MQTT_IN_PASSWORD are used to connect to the MQTT broker, with url authentication (mqtt://MQTT_IN_USERNAME:MQTT_IN_PASSWORD@MQTT_IN_HOST:MQTT_IN_PORT)
  • MQTT_IN_USERNAME: Optional MQTT username
  • MQTT_IN_PASSWORD: Optional MQTT password
  • MQTT_SUBSCRIBE_TOPIC: ezvpn-mqtt topic to subscribe (e.g. fld/modbus/r/#). Defaults to #. Subscribe to multiple topics can be obtained defining each topic on a separated line.
  • MQTT_WANT_QOS2: ask MQTT to upgrade each incoming message on MQTT_SUBSCRIBE_TOPIC to QoS 2 (delivery guaranteed). MQTT will honor the request only if persistence is enabled.
  • SKIP_RETAIN: If true, discard all incoming MQTT messages with retain flag set to true. Defaults to false.

With SKIP_RETAIN set to false, each client that subscribes to a topic pattern receives the retained message immediately after they subscribe.

The broker stores only one retained message per topic.

Destination MQTT connection

  • MQTT_OUT_HOST: MQTT destination host.
  • MQTT_OUT_PORT: MQTT destination port.
  • MQTT_PUBLISH_TOPIC: IOhubTM destination MQTT topic. Everything incoming from MQTT_SUBSCRIBE_TOPIC will be published to MQTT_PUBLISH_TOPIC. If empty, the same incoming topic will be used for publish.
  • MQTT_OUT_QOS: MQTT QoS for published messages. Valid values are 0, 1, 2. Defaults to 0.
  • MQTT_OUT_RETAIN: MQTT retain for published messages. Allowed values are true, false Defaults to false.
  • MQTT_OUT_USE_URL_AUTH: If true MQTT_OUT_USERNAME and MQTT_OUT_PASSWORD are used to connect to the MQTT broker, with url authentication (mqtt://MQTT_OUT_USERNAME:MQTT_OUT_PASSWORD@MQTT_OUT_HOST:MQTT_OUT_PORT)
  • MQTT_OUT_USERNAME: Optional MQTT username
  • MQTT_OUT_PASSWORD: Optional MQTT password

Retry on error

  • RETRY_ON_ERROR: If true, on communication failure, publish is retried every RETRY_INTERVAL milliseconds, forever. Otherwise the message is discarded. Defaults to false.
  • RETRY_INTERVAL: interval in milliseconds between each publish retry. Defaults to 5000. Used only if RETRY_ON_ERROR is true.

Embedded Metadata

  • METADATA: JSON payload. If present, it is added to the standard payload sent to MQTT under the field metadata. Can be any valid JSON type (number, string, boolean, object, array, ...)

Detached Metadata

  • USE_METADATA_CHANNEL: if true, detached metadata is enabled. Defaults to false.
  • METADATA_FOR_CHANNEL: the content of the metadata property, in the payload sent to METADATA_TOPIC. Ignored if USE_METADATA_CHANNEL is not true.
  • METADATA_ID: the metadata instance id. It should be different in each iohub application. Ignored if USE_METADATA_CHANNEL is not true.
  • METADATA_TOPIC: a pubsub topic, receiving the metadata. Ignored if USE_METADATA_CHANNEL is not true.

Message format

  • SEND_PROTOCOL_AS_TOPIC: If true, the protocol value is used as a topic suffix to MQTT_PUBLISH_TOPIC, publishing to a topic named MQTT_PUBLISH_TOPIC/<protocol>/<measurement>, mirroring the incoming topic structure. Otherwise, the protocol is added to the payload using protocol as property name and the measurement is added to the payload using measurement as property name. If MQTT_PUBLISH_TOPIC is empty, the leading / will not be added. Defaults to false.
  • SEND_RAW: If true, message is not interpreted as an internal IOhubTM message format ({ "value": "<value>", "ts": <unix timestamp> }). The messages is forwarded as received. If true, SEND_PROTOCOL_AS_TOPIC is not used.

Message read and write summary

The table below describes, for the different combinations of SEND_RAW, SEND_PROTOCOL_AS_TOPIC, MQTT_PUBLISH_TOPIC:

  • how the output topic and payload are generated
  • the accepted input format

where

  • <a/b/c> is a user defined name for the outgoing topic defined in MQTT_PUBLISH_TOPIC
  • <p> is the incoming protocol, extracted from fld/protocol/r/measurement
  • <m> is the incoming measurement, extracted from fld/protocol/r/measurement
Message read end write summary
Message read end write summary

Docker container details

Image: us-central1-docker.pkg.dev/ez-shared/iohub/iohub-mqtt-mqtt

Supported architecture: amd64

Changelog

v1.0.6
  • Smaller image
  • SEND_PROTOCOL_AS_TOPIC: if true, measurement is now not embedded in the sent payload; it gets added to the topic
  • SEND_RAW: Environment Variable added
v1.0.5
  • MQTT_IN_USE_URL_AUTH: Environment Variable added
  • MQTT_IN_USERNAME: Environment Variable added
  • MQTT_IN_PASSWORD: Environment Variable added
v1.0.4
  • SEND_PROTOCOL_AS_TOPIC: Environment variable added
  • METADATA: Environment variable added
  • USE_METADATA_CHANNEL: Environment variable added
  • METADATA_FOR_CHANNEL: Environment variable added
  • METADATA_ID: Environment variable added
  • METADATA_TOPIC: Environment variable added
v1.0.3
  • MQTT_OUT_USE_URL_AUTH: Environment variable added
  • MQTT_OUT_USERNAME: Environment variable added
  • MQTT_OUT_PASSWORD: Environment variable added
v1.0.2
  • MQTT_SUBSCRIBE_TOPIC accepts multiple topics
v1.0.1
  • MQTT_WANT_QOS2 Environment variable added
  • MQTT_OUT_QOS Environment variable added
  • MQTT_OUT_RETAIN Environment variable added
  • SKIP_RETAIN Environment variable added
  • RETRY_ON_ERROR Environment variable added
  • RETRY_INTERVAL Environment variable added
v1.0.0
  • First Release