The current state of the Internet-of-Things scene can be sometimes mind-boggling: incompatible ecosystems, an endless reliance on cloud services (that will and have been shut down), and an uncomfortable feeling that you’re not quite in control of what your devices are doing. Then I got Home Assistant, and everything changed. This is a story about how smart-home should and shouldn’t be, along with a few tricks that will hopefully save you some blood, sweat, and tears.
The E in IoT stands for easy
If you’re into home automation, then you are probably aware of the absolute mess that the market has become: everything has its own ecosystem, there’s poor integration between services, and everything relies on the cloud all of the time: turning on a switch from 5 meters away usually has to pass through a server hundreds or thousands of kilometers away.
If you want to make an Ecobee thermostat work with a Sonoff temperature sensor, you’re all out of luck. Local network control? nope. Custom hardware? not really. While most of the internet has coalesced thanks to open and flexible standards like IP, DNS, and TCP, the smart home situation has been plagued with the xkcd standards problem: incompatible ecosystems, walled gardens, proprietary protocols, and an overall sensation that companies are prioritizing their own profit margins over trying to create a sustainable market.
Speaking of sustainable, why the hell does everything have to talk to the cloud? There is very little computing power involved to turn on a relay, why do I have to tell Tuya or Google Nest or eWeLink or whoever that I’m turning on my heating, or my room lamps? What will happen when, not if, these companies decide to retire these services? Are we all doomed to be left with useless pumpkins the second this market stops making money?
If you think I’m exaggerating, this has already happened before. Samsung went all in with an entire smart home ecosystem called SmartThings, which claimed, like all of them do, to be your one-stop-shop for all of your home automation needs. This was scaled back after they realized how much of a pain in the ass all of this is to maintain, breaking compatibility with many devices. I still have a v1 SmartThings Zigbee/Z-Wave hub that I cannot use because it’s not supported anymore: the hardware is perfectly fine, but that’s what Samsung decided, so we’re all fucked.
Even well-intentioned endeavors to standardize like Matter, Thread, and Zigbee have all become their own little niche because none of them are actual all-in-one solutions, they are just puzzle pieces: transport protocols, physical networking standards, computing services, whatever. They all have to talk to each other to work, and that is usually left to the end user.
Home Assistant
In comes Home Assistant: an open source project that aims to put this madness to an end. It’s a whole operating system built on Linux, that aims to create a truly complete smart home hub that will communicate with everything. By creating a standard interface for all of your devices to communicate with HA, you essentially get a single interface for all of your smart-home stuff.
We could spend days talking about all of the ins-and-outs of Home Assistant, but this is the gist of it:
- Home Assistant runs on a device with access to your local network: it can be a Raspberry Pi, a virtual machine, or even the custom hardware solutions Home Assistant has to offer.
- Each ecosystem connects to HA via an integration: these blocks connect to APIs or local devices in order to talk to a specific services. There are lots of officially supported ones, as well as some very good community implementations.
- Integrations usually generate devices, which in turn generate entities. An entity is the base building block of Home Assistant: every sensor, switch, light, or whatever you want is represented as an entity. Entity data is stored on a local database and can be shown on custom dashboards.
- Integrations also generate actions, which involve executing procedures on command, like refreshing data, actuating a switch, turning on a lamp, etc. There are standard integrations for the more common entities, but there are endless possibilities.
- You can create custom entities called helpers, which further process information given by your entities. These are also entities themselves.
- Entities can be analyzed and processed using automations, which run instructions based on the state of different entities. These instructions are actions executed on devices or entities.
Overall, it’s a fairly simple system, but it’s also highly scalable: you can make this as complicated as you want, as long as you follow these basic rules. Do you want to turn on lights with the sun? Make an integration that triggers using the sun entity during sunrise and sunset, that actuates the light switch. Do you want to make an automated sprinkler system? Sure, make helpers for all parameters and have an automation switch relays for sprinkler valves in order. Do you want to fire a notification to your phone when a temperature sensor is under 19.2 degrees but only during the evening and there are no dishes on your dishwasher? Sure, but I don’t know why you would, although I’m not here to judge. As long as it has some sort of connectivity, chances are, you can make it work with Home Assistant.
Why isn’t this the norm?
Well, there are complications to this, mostly stemming from the fact that Home Assistant isn’t exactly maintenance free: You need to have a device running the OS locally, which will require some tinkering. Also, as your instance becomes more complicated, your points of failure also sprawl, and while HA is overall fairly stable, it does throw the occasional tantrum.
There are also some companies that have obfuscated open access to their APIs in the name of “customer safety”. This is usually a measure to make their ecosystems even more of a walled garden, so I would recommend just avoiding these products: they are not in favor of right-to-repair, and I frankly have no sympathy for them. I do however recommend you to look around for custom integrations that give increased functionality in case you’re stuck with such a device.
There are also many finicky steps to get some integrations to work: handing over login credentials is one thing, but that’s sometimes not enough; Sometimes you need API keys, sometimes OAuth tokens, sometimes other stuff. These things are usually well-documented, but they are usually buried under layers of menus and interfaces that often feel like afterthoughts. The worst one for me is LocalTuya, a custom integration for Tuya devices that allows for, admittedly, a very useful increase in functionality over the official Tuya integration, but it requires many steps to get the API to work, and the entities have to be set up by hand, without much in the way of help. I only have a single device at the moment, but configuring 12 entities in the blind was an absolute nightmare, and my stomach turns a little when I think of adding more devices.
I also have some issues with using SQLite as a main database. Sure, it’s easy and fairly uninvolved to get working, but my Home Assistant database has corrupted itself one too many times for me, without much in the way of me doing anything. I switched to my local MySQL server about a month ago and my database has been much more reliable since.
Making custom devices: ESPHome
If you’re weird like me (and let’s face it, if you’re reading this you already are), you probably want to plug everything into Home Assistant, even those things that weren’t really meant to be connected to a network. For me, it was a pool, but it can be anything that has sensors and/or actuators of some kind.
I have a pool with a solar thermal collector, a lamp, and a copper electrolytic cell. This means we have quite a few variables we would like to integrate to HA:
- Temperature values for input, panel return, and output pipes. This not only allows me to get the current pool temperature, but also to get rough estimates about panel power capture.
- Actuating relays for turning on the pool pump and lamp.
- Controlling an H-bridge connected to the copper cell, allowing for control of voltage and polarity of the cell.
Obviously there isn’t a device that can do exactly this, and while you could make it work with various devices it would be a janky mess, so actually building a custom IoT device actually makes some sense here.
In comes ESPHome: using the Espressif ESP32 or ESP8266 family of microcontrollers, Home Assistant can create custom devices using simple YAML config files and a ton of supported devices. Just connect the sensors and devices you want to the ESP, set everything up on the config file, and that’s it! you have your own IoT device, and a surprisingly flexible one at that: sensors can be filtered, automations can be configured directly on board the controller, and so much more. I plan on doing a detailed review of the ESPHome suite at another time, but suffice it to say that it allows to make absolutely anything that can be reasonably connected to a microcontroller HA-capable.
Why hasn’t anyone made this simpler?
It boggles my mind a little bit that no one has thought of a more hands-off approach to this. For most intents and purposes, a hardened Home Assistant brick that can just sit in a corner and get online via the cloud subscription is enough for your usual tech enthusiast.
Nabu Casa, Home Assistant’s main developers, have already started offering plug-and-play hardware solutions in its Green and Yellow packages, but in my opinion there are way too many growing pains in HA for it to be a truly fire-and-forget solution: there is way too much tinkering still if you want to do everything HA is capable of.
So I wonder, why hasn’t it been a push to standardize IoT ecosystem interactions between different brands? Why have pretty much all IoT brands gone for any sort of interoperability? Well, money is the reason, but I wonder if this is a losing strategy: what’s the point of having three thousand apps to turn on an A/C unit and a couple of lamps? How does this nightmare of usability not impact sales of these systems?
While it probably does, the answer is they don’t really care: they sell enough to be a revenue stream, but the extra work that would create actually making a sustainable product is just too much upfront cost to justify it: in the end, all sustainable IoT ecosystems are passion projects: open-source and free software which challenges market incentives. There is this undercurrent of skepticism in my writing about tech, but it comes down to this: the market supports whatever is cheaper, not whatever is best, and there are consequences to be had if the tech sector keeps going after it at all costs.
Closing thoughts
If you’re the tinkering type like me and you haven’t set up something like Home Assistant, please do! It’s genuinely useful and quite fun, but be prepared for a bit of head-banging: it’s not for free. It’s now an essential part of my home, and it provides amazing data collection and intelligent operation, allowing for increased efficiency and automation, even if it came at the price of many hours of staring at YAML config files, and corrupted logs.
So please, if you work in this sector, remember what has made our Internet work: open, flexible standards which work everywhere and with everything. If these principles can be applied to IoT, I am confident in saying that IoT will be a mass-appeal product.