Home automation with MQTT and Home Assistant — TechTalk Gist (2023)

Summary of my lecture on technology for students at Moratuwa University (CSE), Sri Lanka

While there are several protocols used by proprietary developers like Google, Amazon, and Philips, in this article we will discuss using MQTT in home automation. This is because MQTT is a trusted open source message brokering framework for IoT communication.

Some tips about MQTT

• It's not a message queuing framework, it's a message brokering mechanism. The queue happens to support the arbitration process, hence the term telemetry.
• IBM scientists invented the protocol in 1999 to monitor a pipeline crossing a desert.
• It uses an open standard, so getting licenses is easy!

How MQTT works

• MQTT uses the publish-and-subscribe or broker architecture.
• We have two main parties; clients and the agent (usually an agent).
• The most popular concealer is Mosquitto (Dispose)
• The MQTT browser can be used to communicate with the broker (MQTT Explorer | A complete MQTT client that offers a structured overview of the topic)
• It uses the TCP/IP stack for communication.

Suppose we have a light in the dining room that is an MQTT client. In addition, there is an energy monitor that monitors the presence of people in the dining room and regulates unnecessary energy consumption. In this example, we'll use the Mosquitto command line tool for simplicity.

The lighting control must subscribe to all incoming updateslight in the dining room.

• Subscribe to the topichouse/light/dining room_living room/#

The energy monitoring system must sign off on everything related to itdining room. There can be PIR sensors, heating/cooling systems, SmartTVs, lamps, etc.

• Subscribe to the topicStart/+/dining room_living room/#

For this we must execute the following commands.

• moskito_sub -v -t house/light/dining room_living room/#
• moskito_sub -v -t home/+/room_jantar/#

Note that we use some wildcards and command line arguments as follows;

• -v: Allows the topic to be viewed, otherwise only the payload is displayed
• -T: Mention the topic
• +: wildcard character to be used in the middle of a topic separator
• #: to use at the end of the topic

We can send the messages to the topic.Home/Light/Dining Room/Setwith the MQTT browser.

Messages are received by command line signing as follows. Please note that we subscribe to a wildcard topic starting withhouse/light/dining room_living room/#what makes up for my mistake to havedining roomtwice in the screenshot above (that was a mistake in my speech, sorry!).

Service quality

Service quality 0

Forget shooting and focus. The MQTT broker doesn't care if all subscribers have received the message or not.

For example:

• DHT sensors (Digital Humidity Temperature) that transmit weather information every 10 s
• Presence sensors (PIR) that check presence every 5 seconds and report status
• Suitable for scenarios where the loss of a single message or data point is not a concern. Or performance is limited and network overhead is kept to a minimum.

Service Quality 1

Messages are guaranteed to be sent at least once. This implies that messages can be sent multiple times.

• Suitable if you expect to get all readings lossless. This also assumes that in this case the duplicate data can be removed in other ways (using timestamps, etc.).
• Climate monitoring where time series information is crucial. Packet loss may occur due to network problems. Therefore, application layer delivery confirmations are required.

quality of service 2

The delivery is guaranteed as a one-off delivery. No duplicate or missing messages.

• Parking garages must monitor the number of vehicles entering in order to show customers available parking spaces. You can enter vehicles from several locations at the same time. No assumptions can be made about time.
• Deep sea surveillance systems where timestamps may not be reliable. In these cases, the messages must be received as often as the events occur.

In QoS 1 and 2. Clients queue messages for delivery, which is often a poor choice for memory-constrained devices like Arduinos or ESPs.

Network usage and power consumption

Network connections can be reliable, but IoT devices can be in states that prevent them from maintaining a stable connection. For example, device fabrics (like ZigBee or ESP) can change topology when moving. Some devices may reboot periodically due to OTA (over-the-air updates). Or the user can decide to turn off the smart lamp in the dining room and bring it to the kitchen!

In such cases, there needs to be a means by which the delivered messages can be made persistent. In other words, if a light comes on and the mains goes down for some reason, the bulb should still be able to turn on when it comes back online. We achieve this by using the hold flag.

By enabling the hold flag, the broker keeps the last message with the hold flag indefinitely. This can be removed with the -remove-retained argument.

The flow of a typical MQTT device is as follows.

• Click the button in the smart home app to turn on L1.

• Send the following payload to the MQTT broker in the threadarduino-esp/luz/L1/set

{
"Status": "ON",
"Brightness": 255
}

• Now the message is sent to the Subscriber (or Controller) Bulb on the same topic with the same payload.
• Now the device brings the lamp relay to a closed position (lamp turns on) and sends the next charge to the wireArduino-ESP/Licht/L1/Status.

{
"Status": "ON",
"Brightness": 255
}

When received by the smart home app, it changes the color of the button to yellow and the brightness level to the set value.

Why is this done?

There can be multiple users controlling the brightness/light. At the same time, this process can be subject to errors due to the many issues we have discussed. Therefore, the lamp driver itself must share its state with the corrector to be safe.

One way to do this is to post the status every 10-20 seconds. And to respond faster, an instant message can be sent when the status changes. Periodic messages can be expensive and prevent devices from going into deep sleep mode. We can only use instant thanks for updates. But for that we need a QoS greater than 0. Otherwise the control application might not respond. This represents a compromise between the device's activity level and its ability to sleep soundly with WiFi or some other type of service.

Steps you must take before you start implementing this example!

• Install the Home Assistant
• Install the MQTT Checker plugin. Shortcut:MQTT
• ESP32 with Arduino IDE and PubSub library (MQTT library)
• ESP32 MQTT Publish-Subscribe mit Arduino IDE(image source + tutorials)
• Try the following!

• A more elegant approach with Home Assistant! Have good UI and mobile apps. It's also open source!

residential assistantYou can automatically discover MQTT-enabled devices and register in the application.MQTT discovery

Detection message template for a light

{
"~": "Home/Light/Kitchen Tools",
"Name": "Kitchen",
"unique_id": "Kitchen light",
„cmd_t“: „~/conjunto“,
„stat_t“: „~/status“,
"schema": "json",
"brightness": true
}

„~“is a placeholder to place its instance in the values ​​of this JSON object.

This message should be sent to Home Assistant's discovery thread, which reads:

Housekeeping/Light/Kitchen/Config

Everything in the JSON object is configurable by the device itself. The theme of discovery must be strictly adhered to. If not, some tricks are required.

Steps:

• The (lightweight) device posts the above JSON to the detection topic.
• Home Assistant adds a lamp control to the user interface
• Change the name to whatever you like, say "dining room light".
• The lightbulb above also has a brightness control and is powered by a JSON payload.
• When the user clicks the button to change the lightbulb, the initial wizard posts a payload to the ~/set topic (determined by cmd_t, which expands to command_topic). "~" is replaced with "Home/Light/Kitchen Assistant".
• The light will then send its new status to ~/status, which is determined by stat_t expanding to theme_state.
• JSON keys are truncated as some devices operate with limited memory and string processing power. It also saves network packet usage, required bandwidth, RAM usage, and internal buffer sizes. Some networks have limited packet sizes or MTUs (Maximum Transmission Units).

To install:Home Assistant Community-Plugin: Node-RED

NodeRED is a virtual connection tool for MQTT (or other) messages. You can convert an MQTT message into one or more other messages. An example to follow.

• PIR sensors detect the empty house for 5 hours.
• Turn off all electrical devices except essential ones (refrigerator, cameras, etc.) (status: BLOCKED)
• The door sensor now recognizes an open message from the user
• Exit the initial LOCK state as follows
• Turn on the garage light
• Turn on the air conditioner and cool the house.
• turn on Geezer
• Open the bedroom blinds to let in the sunlight.
• play some music
• If necessary, the EXIT BLOCK routine can take a different route, for example if someone comes home through the front door without a car. All this is 100% compatible with Home Assistant and can be done with Arduino IDE and ESP8266 or ESP32 MCU.

1. Privacy is a major concern in the modern world.
2. Companies like Ubiquiti offer affordable Wi-Fi solutions for whole homes. This means that the limitations of devices such as Philips Hue, Google Home or Alexa are no longer a problem. Either way you have WiFi.
3. ZigBee devices are expensive and of limited variety. (Philips Hue bulb: Rs 5000+ per bulb). It's a closed pattern. However, the ESP32 costs less than Rs 2000 (even cheaper at Ali) and can automate a huge chandelier with little to no effort.
4. No internet access required. Home Wizard was released as the operating system for the Raspberry. The open-source mobile app (Android and iOS) can be run on the local network. The quality and stability of these apps is much better than Google and Alexa.

• It is a SPOF (single point of failure).
• MQTT needs to run with a watchdog (to be restarted if it fails).
• Misconfigured MQTT can be vulnerable to intruders.
• MQTT brokers must always be online. This is not the case with BLE signal heads. So it has to run on a mains-powered device.
• Routing is based on IP. Basically, this means that only a limited number of devices can connect on a given subnet. You need a DHCP server to assign IP addresses. This is not the case with ZigBee or ESP mesh.
• It is not considered suitable for IIoT (Industrial IoT) applications. One of the main reasons is the QoS support. QoS is only taken into account when subscribers and publishers are at the same QoS level. In MQTT, the same topic can have multiple subscribers with different QoS levels. This makes delivery more difficult for the publisher and the effort for communication is high.

File:Is MQTT the answer to IIoT?

• This is an application layer messaging protocol. This makes it difficult to forward data in a personalized way. For example, you cannot reach a remote device without a Wi-Fi connection by asking a nearby device to transfer data. However, there are network layer messaging applications that support WiFi mesh networks that go beyond that. (Painless Mesh Arduino)

• Protection with SSL (MQTTS)
• Use bridge devices connecting to MQTT brokers.
• The bridge device can connect other devices and transmit messages between external devices and the mesh. The Philips Hue Bridge does something similar, connecting IP network and ZigBee lightbulbs. This can be fully achieved withowagner/hue2mqtt - Gateway between a Philips Hue Bridge and MQTTand the official Philips Java API library.

This is really a bulky article. However, I think this can also be complete for someone looking to get into home automation with Home Assistant.