table of contents
msoak(1) | MQTT utilities | msoak(1) |
NAME¶
msoak - consume messages from arbitrary MQTT brokers and subscriptions
SYNOPSIS¶
msoak [ -v ] configuration
DESCRIPTION¶
msoak is a utility with which to simultaneously subscribe to an arbitrary number of topics on any number of MQTT brokers and optionally modify or normalize received payloads before printing them out. This utility was created to back up to a central location messages received by a hand full of brokers; instead of launching (and having to monitor success of) a large number of mosquitto_sub(1) programs, msoak took on the job.
msoak uses asynchronous connects to the MQTT brokers so that it can handle situations in which a broker may temporarily be unavailable even when msoak initially starts, and msoak embeds a Lua interpreter for which you can create scripts which act on messages received to, for example, trigger actions.
msoak can be used with, say, tinylog(8) to create compressed archives of data.
msoak my.config | tinylog -k 30 -s 1077936128 -z archive
OPTIONS¶
- -v
- print version information and exit.
CONFIGURATION¶
msoak reads the configuration file using libconfig(3) utility functions. This is a short example:
# this is a comment luascript = "example.lua" servers = (
{ id = "local"
host = "localhost"
port = 1883
fmt = "greet"
showid = true
showtopic = true
topics = [
"tests/sponge/#",
"some/thing/else/+/+"
]
},
{ id = "tmo"
host = "test.mosquitto.org"
qos = 0
showid = true
showtopic = true
topics = [
"home/special",
"nothome/dont"
]
}
);
The configuration consists of global settings and a list or array of servers, each with a number of settings.
global variables¶
server array¶
- id
- identifier for the connection, also shown on output. If id is unset we generate a numeric id.
- host
- host name or address of the MQTT broker. Defaults to localhost.
- port
- the TCP port number for the MQTT broker. Defaults to 1883.
- clientid
- MQTT clientId to use. The client identifier defaults to the string msoak-hostname:basename(configfilename) so that the program can run on multiple hosts with the same configuration.
- qos
- MQTT QoS (default 1)
- user
- username for the MQTT connection
- pass
- password for the MQTT connection; see passenv to omit from having to specify clear-text passwords in the configuration file.
- passenv
- name of environment variable (including $) from which to obtain pass.
- showretained
- consume retained messages; default true
- cacert
- path to PEM-encoded CA certificate chain for TLS connections to MQTT broker
- showid
- true or false, default true; whether to print id on output.
- showtopic
- true or false, default true; whether to print topic name on output.
- topics
- array of strings with topic branch names for msoak to subscribe to.
- fmt
- optional name of Lua formatting function (see below).
Formatting¶
By default, the received payload is printed to standard output, optionally prefixed by the message topic (if showtopic is true), and it is preceeded by the connection identifier if showid is true.
If fmt is set, it contains a string with the name of a Lua function from the Lua script specified in the luascript global. this function is used to format the payload msoak receives; the return value of the function replaces the original payload and is printed out.
If msoak can decode the payload into JSON (i.e. the message begins with a brace ({) and the JSON can be decoded), it will invoke the Lua function to obtain output.
function init()
if msoak.verbose then
msoak.log(msoak.luascript .. " starting up")
end end function greet(topic, _type, d)
s = string.format("Hello %s -> now=%s", d.name,
msoak.strftime("%TZ"))
return s end function exit()
msoak.log("Hasta la vista, baby!") end
The optional init() and exit() functions are invoked when msoak begins and ends respectively. The greet function is invoked for each message for which a server in the configuration file contains fmt = "greet".
Lua functions¶
There are a few variables and functions msoak implements which are available to the Lua scripts you use.
- version
- returns the msoak version number as a string
- luascript
- returns the file name of the luascript global variable
- verbose
- is a boolean which indicates whether msoak is running in verbose mode
- msoak_log()
- accepts a string which is printed to stderr prefixed by "MSOAKLOG:".
- msoak_strftime()
- expects a format string and integer seconds and implements strtime(3) for Lua with the specified format and seconds and returns the string result to Lua. As a special case, if seconds is less than one it uses current time.
JSON¶
When configured to use a Lua script file, msoak attempts to decode incoming JSON payloads and will pass the decoded JSON elements to the configured fmt function as a table with these additional elements in it
- _conn_id
- the original connection id
- _conn_host
- the hostname of the connection
- _conn_port
- the port number of the connection
- _conn_topic
- the MQTT topic on which the original payload message was received
Note that if no luascript was specified and the payload contains JSON it will be dumped as is to stdout.
ENVIRONMENT¶
Any number of environment variables may be used by msoak if specified in passenv settings with the configuration.
BUGS¶
What's with the strange name? Just as I started working on this program I learned about sponge(1) and loved the name. The rest is history.
Note that there are different versions of libconfig(3) floating around which may have effects on the syntax permitted in msoak's configuration file .
AUTHOR¶
Jan-Piet Mens, https://github.com/jpmens/msoak
SEE ALSO¶
mosquitto_sub(1), mqttwarn, sponge(1), strftime(1), tinylog(8)
January 2020 | jpmens |