Scroll to navigation

msoak(1) MQTT utilities msoak(1)

NAME

msoak - consume messages from arbitrary MQTT brokers and subscriptions

SYNOPSIS

msoak [ -v ] configuration

DESCRIPTION

msoak is a utility with which to simultaneously subscribe to an arbitrary number of topics on any number of MQTT brokers and optionally modify or normalize received payloads before printing them out. This utility was created to back up to a central location messages received by a hand full of brokers; instead of launching (and having to monitor success of) a large number of mosquitto_sub(1) programs, msoak took on the job.

msoak uses asynchronous connects to the MQTT brokers so that it can handle situations in which a broker may temporarily be unavailable even when msoak initially starts, and msoak embeds a Lua interpreter for which you can create scripts which act on messages received to, for example, trigger actions.

msoak can be used with, say, tinylog(8) to create compressed archives of data.

msoak my.config | tinylog -k 30 -s 1077936128 -z archive

OPTIONS

-v
print version information and exit.

CONFIGURATION

msoak reads the configuration file using libconfig(3) utility functions. This is a short example:


# this is a comment luascript = "example.lua" servers = (
{ id = "local"
host = "localhost"
port = 1883
fmt = "greet"
showid = true
showtopic = true
topics = [
"tests/sponge/#",
"some/thing/else/+/+"
]
},
{ id = "tmo"
host = "test.mosquitto.org"
qos = 0
showid = true
showtopic = true
topics = [
"home/special",
"nothome/dont"
]
}
);

The configuration consists of global settings and a list or array of servers, each with a number of settings.

global variables

the path to the Lua script file to use. If left unset, no Lua processing is performed.
whether or not to be verbose (default true)

server array

identifier for the connection, also shown on output. If id is unset we generate a numeric id.
host name or address of the MQTT broker. Defaults to localhost.
the TCP port number for the MQTT broker. Defaults to 1883.
MQTT clientId to use. The client identifier defaults to the string msoak-hostname:basename(configfilename) so that the program can run on multiple hosts with the same configuration.
MQTT QoS (default 1)
username for the MQTT connection
password for the MQTT connection; see passenv to omit from having to specify clear-text passwords in the configuration file.
name of environment variable (including $) from which to obtain pass.
consume retained messages; default true
path to PEM-encoded CA certificate chain for TLS connections to MQTT broker
true or false, default true; whether to print id on output.
true or false, default true; whether to print topic name on output.
array of strings with topic branch names for msoak to subscribe to.
optional name of Lua formatting function (see below).

Formatting

By default, the received payload is printed to standard output, optionally prefixed by the message topic (if showtopic is true), and it is preceeded by the connection identifier if showid is true.

If fmt is set, it contains a string with the name of a Lua function from the Lua script specified in the luascript global. this function is used to format the payload msoak receives; the return value of the function replaces the original payload and is printed out.

If msoak can decode the payload into JSON (i.e. the message begins with a brace ({) and the JSON can be decoded), it will invoke the Lua function to obtain output.


function init()
if msoak.verbose then
msoak.log(msoak.luascript .. " starting up")
end end function greet(topic, _type, d)
s = string.format("Hello %s -> now=%s", d.name,
msoak.strftime("%TZ"))
return s end function exit()
msoak.log("Hasta la vista, baby!") end

The optional init() and exit() functions are invoked when msoak begins and ends respectively. The greet function is invoked for each message for which a server in the configuration file contains fmt = "greet".

Lua functions

There are a few variables and functions msoak implements which are available to the Lua scripts you use.

returns the msoak version number as a string
returns the file name of the luascript global variable
is a boolean which indicates whether msoak is running in verbose mode
accepts a string which is printed to stderr prefixed by "MSOAKLOG:".
expects a format string and integer seconds and implements strtime(3) for Lua with the specified format and seconds and returns the string result to Lua. As a special case, if seconds is less than one it uses current time.

JSON

When configured to use a Lua script file, msoak attempts to decode incoming JSON payloads and will pass the decoded JSON elements to the configured fmt function as a table with these additional elements in it

_conn_id
the original connection id
_conn_host
the hostname of the connection
_conn_port
the port number of the connection
_conn_topic
the MQTT topic on which the original payload message was received

Note that if no luascript was specified and the payload contains JSON it will be dumped as is to stdout.

ENVIRONMENT

Any number of environment variables may be used by msoak if specified in passenv settings with the configuration.

BUGS

What's with the strange name? Just as I started working on this program I learned about sponge(1) and loved the name. The rest is history.

Note that there are different versions of libconfig(3) floating around which may have effects on the syntax permitted in msoak's configuration file .

AUTHOR

Jan-Piet Mens, https://github.com/jpmens/msoak

SEE ALSO

mosquitto_sub(1), mqttwarn, sponge(1), strftime(1), tinylog(8)

January 2020 jpmens