Categories
Internet of Things

MQTT – Internet of Things Messaging with Mosquitto

MQTT (Message Queue Telemetry Transport) is a messaging wire communications protocol for machine to machine (M2M) and software component integration.

Use cases:

  • Sensors / IOT devices
  • Robotics, Industrial Monitoring & Remote Control
  • Automation – Smart Home
  • Mobile, Web Browser UI clients, Cloud Services
  • Integration – local & long range communication between software & services

In contrast to world wide web, where browsers pull web pages from internet servers (request/response), MQTT implements a Publish / Subscribe (PubSub) and a push messaging model.

Common with HTTP at transport OSI network layer MQTT extends TCP/IP.

MQTT clients connect and transmit to brokers (servers), who register subscribers to named topics (channels).

A WiFi enabled ESP8266 device and Mobile (Android) interface

Message data payload can contain arbitrary binary or UTF-8 Unicode encoded text data (up to 256 MB per message).

Internet of Things (IOT) devices deploy MQTT as a lightweight email (SMTP) like solution for data exchange suitable for integrating software, sensors, devices and cloud services.

An OASIS standard currently at version 5x ( http://mqtt.org/ ) defines capabilities including:

  • Distributed, bi-directional message flows
  • Quality of Service (QOS) delivery levels: at least once, only once, guaranteed message order
  • Security: Authentication, Encryption – TLS, OpenSSL, OAuth
  • Low power consumption / bandwidth
  • Push-store-forward including to offline clients
  • Browser client integration with WebSockets (RFC 6455)

Lets take a closer look at Mosquitto ( https://mosquitto.org/ ) an open source MQTT broker implementation sponsored by Apache Software Foundation .

Mosquitto MQTT Setup under Linux Ubuntu

Mosquitto can be installed on Ubuntu with package manager:

sudo apt-get install mosquitto mosquitto-clients -y  

From a binary package ( https://mosquitto.org/files/source/ ) or compiled from source:

Download source:

mkdir mqtt
cd mqtt
git clone https://github.com/eclipse/mosquitto

Install Dependencies:

sudo apt-get install git cmake libc-ares-dev uuid-dev libssl-dev zlib1g-dev xsltproc docbook-xsl
RFC6455 WebSockets
sudo apt-get install libwebsockets-dev
Unit Test Framework
sudo apt-get install libcunit1 libcunit1-doc libcunit1-dev

Edit build config (eg add websockets)

vi config.mk

Build / Install

make clean
make test
make
make install

Mosquitto File Paths / Commands:

Config File
/etc/mosquitto/conf.d/default.conf

Logs
/var/log/mosquitto/mosquitto.log

Start/stop/restart

sudo /etc/init.d/mosquitto restart
[ ok ] Restarting mosquitto (via systemctl): mosquitto.service.
1594324669: Config loaded from /etc/mosquitto/mosquitto.conf.
1594324669: Opening ipv6 listen socket on port 1883.
1594324669: Opening ipv4 listen socket on port 1883.
1594324669: Opening ipv4 listen socket on port 8883.
1594331039: mosquitto version 1.6.10 starting

Mosquitto Password Authentication

Mosquitto user/password authentication can be required on a per listener basis.

Add broker authentication per listener in default.conf

listener 1883 localhost
allow_anonymous false
password_file /etc/mosquitto/passwd

Generate per user password credential –

sudo mosquitto_passwd -c /etc/mosquitto/passwd <user>

To prevent disclosure of password in shell command history an environment variable can be set in user profile –

MOSQUITTO_PASSWORD=<password>

Command line connect string references environment password variable

mosquitto_pub -h localhost -t test.out -m "test message #3" -u mqtt -P $MOSQUITTO_PASSWORD

Mosquitto SSL setup

Mosquitto man page has information on SSL and TLS setup.

Firewall port 8883 is opened (production deployment should restrict connection to known clients)

sudo ufw allow from any to any port 8883 proto tcp

To use self signed SSL certificates we create a Certificate Authority (CA) key –

sudo openssl req -new -x509 -days 360 -extensions v3_ca -keyout ca.key -out ca.crt

Next we generate and sign certificate for MQTT server, noting that key encryption is not supported –

sudo openssl genrsa -out server.key 2048
sudo openssl req -out server.csr -key server.key -new
sudo openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 360

Finally we create a certificate for MQTT client –

sudo openssl genrsa -des3 -out client.key 2048
sudo openssl req -out client.csr -key client.key -new
sudo openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out  client.crt -days 360

Key Points –

  • Server key must not use key encryption or Mosquitto raises error:
	1594334396: Error: Unable to load server key file "/etc/mosquitto/certs/server.key". Check keyfile.
	1594334396: OpenSSL Error[0]: error:2807106B:UI routines:UI_process:processing error
	1594334396: OpenSSL Error[1]: error:0906406D:PEM routines:PEM_def_callback:problems getting password
	1594334396: OpenSSL Error[2]: error:0906A068:PEM routines:PEM_do_header:bad password read
	1594334396: OpenSSL Error[3]: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib

  • Common Name (CN) must be different across each of CA, Server and Client certs for example –

Common Name (e.g. server FQDN or YOUR name) []:
CA: selfcert.local
Server: mqtt.steveio.com
Client: ESP32

Otherwise following errors occur –

Server:
1594327583: OpenSSL Error: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca

Client:
stevee@ideapad-530S:~/ssl_cert$ sudo mosquitto_pub --insecure --cafile /etc/mosquitto/ca_certificates/ca.crt --cert /etc/mosquitto/certs/client.crt --key /etc/mosquitto/certs/client.key -h mqtt.steveio.com -p 8883 -q 1 -t "test" -i anyclientID --tls-version tlsv1.3 -m "Hello" -d
Enter PEM pass phrase:
Client anyclientID sending CONNECT
OpenSSL Error[0]: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
Error: A TLS error occurred.

Mosquitto Broker SSL Configuration

Mosquitto Broker SSL default.conf Listener Configuration

listener 8883 mqtt.steveio.com
cafile /etc/mosquitto/ca_certificates/ca.crt
keyfile /etc/mosquitto/certs/server.key
certfile /etc/mosquitto/certs/server.crt

Restart Mosquitto service and check logs

sudo service mosquitto restart
sudo cat /var/log/mosquitto/mosquitto.lo

1596376328: mosquitto version 1.6.10 starting
1596376328: Config loaded from /etc/mosquitto/mosquitto.conf.
1596376328: Opening ipv6 listen socket on port 1883.
1596376328: Opening ipv4 listen socket on port 1883.
1596376328: Opening ipv4 listen socket on port 8883.

Testing SSL connection with openSSL client

# use openssl client to test connection
sudo openssl s_client -connect mqtt.steveio.com:8883 -CAfile /etc/mosquitto/ca_certificates/ca.crt

SSL Publish using Mosquitto broker command line

sudo mosquitto_pub --cafile /etc/mosquitto/ca_certificates/ca.crt --cert /etc/mosquitto/certs/client.crt --key /etc/mosquitto/certs/client.key -h mqtt.steveio.com -p 8883 -q 1 -t "test" -i anyclientID --tls-version tlsv1.3 -m "Hello" -d -u mqtt -P $MOSQUITTO_PASSWORD 

Enter PEM pass phrase:
Client anyclientID sending CONNECT
Client anyclientID received CONNACK (0)
Client anyclientID sending PUBLISH (d0, q1, r0, m1, 'test', ... (5 bytes))
Client anyclientID received PUBACK (Mid: 1, RC:0)
Client anyclientID sending DISCONNECT

SSL Subscribe using Mosquitto broker

sudo mosquitto_sub --cafile /etc/mosquitto/ca_certificates/ca.crt --cert /etc/mosquitto/certs/client.crt --key /etc/mosquitto/certs/client.key -h mqtt.steveio.com -p 8883 -q 1 -t "test" -i anyclientID --tls-version tlsv1.3 -d -u mqtt -P $MOSQUITTO_PASSWORD

MQTT Offline Clients – Store / Persist / Forward

A feature within MQTT specification is store / forward, enabling message delivery to offline or clients connecting only on a scheduled basis. Normally, messages are delivered immediately to all topic subscribers. When clients setup a persistent session, messages with QOS level 1 or 2 are queued for delivery in the event they are offline.

Requirements:

  • QoS level: 1 / 2
  • Fixed client ID
  • Always connect with clean_session=False
  • Subscriptions must be made with QoS>0
  • Messages published must have QoS>0

Example Store / Forward Message Flow

Subscriber specifies qos level 1, client identity (-i), clean flag (-c)

mosquitto_sub -h localhost -t test.out -u mqtt -P $MOSQUITTO_PASSWORD -q 1 -c -i broker

Client -c5646 sending CONNECT
Client -c5646 received CONNACK
Client -c5646 sending SUBSCRIBE (Mid: 1, Topic: test.out, QoS: 1)
Client -c5646 received SUBACK
Subscribed (mid: 1): 1

Publisher sends msg #1 to online client

mosquitto_pub -h localhost -t test.out -m "test message #1" -u mqtt -P $MOSQUITTO_PASSWORD -d -q 1 -i client

Client -c8212 sending CONNECT
Client -c8212 received CONNACK
Client -c8212 sending PUBLISH (d0, q1, r0, m1, 'test.out', ... (15 bytes))
Client -c8212 received PUBACK (Mid: 1)
Client -c8212 sending DISCONNECT

Subscriber (online) receives message #1

Client -c8186 received PUBLISH (d0, q1, r0, m1, 'test.out', ... (15 bytes))
Client -c8186 sending PUBACK (Mid: 1)
test message #1

Subscriber disconnects, publisher sends message #2

mosquitto_pub -h localhost -t test.out -m "test message #2" -u mqtt -P $MOSQUITTO_PASSWORD -d -q 1 -i client

Subscriber connects, receiving message sent during offline

mosquitto_sub -h localhost -t test.out -u mqtt -P $MOSQUITTO_PASSWORD -d -q 1 -c -i broker

Client broker sending CONNECT
Client broker received CONNACK
Client broker sending SUBSCRIBE (Mid: 1, Topic: test.out, QoS: 1)
Client broker received PUBLISH (d0, q1, r0, m2, 'test.out', ... (15 bytes))
Client broker sending PUBACK (Mid: 2)
test message #2
Client broker received SUBACK
Subscribed (mid: 1): 1

There is a more detailed examination of persistence and store / forward here.

As MQTT matures it would be useful to see tools like Postfix qshape emerge for bottleneck and message queue administration.

By admin

Software and Internet developer with an open source perspective interested in web, coding, data analytics & visualisation, micro-controllers, internet of things, sensor devices, networks and electronics.

Leave a Reply

Your email address will not be published. Required fields are marked *