At Crownstone we incorporate artificial intelligence or more old-fashioned machine learning techniques in our products to create really smart homes. A smart home does things for you automatically. A smartphone app that only provides you with yet another option to control a light is not what Crownstone is after. It would mean that you have to unlock your phone, open the right app, navigate to the right screen, and click the right button. That can be accomplished much easier by just flipping a light switch. No, we want that your home responds to your presence without you having to do anything!
For that we develop resource-constrained artificial intelligence (AI) which resides on the embedded chips on the Crownstones themselves - plus taking into consideration bandwidth and latency - on the smartphones, and in the cloud. If you just have bought a Crownstone system, one of the first interactions with the AI is to give it a name (see the screenshot at the right). This is all cool, but what are the capabilities of the AI that we are developing? There are multiple research directions:
Most technological solutions use the smartphone to localize itself. However, there is another, better way to do localization if you have a network of Crownstone available. That is to use the Crownstones themselves to pinpoint the location of a person with a smart device. This has the following advantages:
Our current research on indoor positioning does not yet address in-network localization:
The Crownstones measure current and voltage simultaneously at high frequencies. If an appliances is plugged into a Crownstone the current curve is not a perfect sine wave. The curve is distorted depending on the type of appliance plugged in. This distortion can subsequently be used to identify that particular device.
In the above thesis you see that deep learning is used to perform appliance identification. Here below you see for example the current consumption of a laptop over a single curve on 50 Hz (which means 20 ms). You see that it is quite spiky, quite different from a normal resistive load! The PLAID data set is also a very good source for high resolution current and voltage data (30 kHz) for a set of 11 devices.
The reason to have appliance identification is manifold:
A very futuristic scenario is the identification of the presence of people while they do not carry a device whatsoever. Due to the disturbances caused by people who absorb part of the electromagnetic waves the 2.4GHz spectrum (of Wi-Fi and Bluetooth Low Energy) gets distorted. Our first preliminary experiments (see thesis below) demonstrate that it is indeed possible to recognize the presence of a person. It is indeed preliminary. A person needs to walk in between two Crownstones breaking a virtual beam.
It is quite promising that recently researchers from the University of California used ordinary Wi-Fi to accomplish indoor positioning as well. Check this medium post on the techniques they used. So, what would be the reason to have device-free presence detection?
Even without Crownstones it is possible to use information about device use, light use, and other patterns to make your home smarter. This is especially useful for elderly people. Lights, coffee machine, vacuum, etc. can all be used like normal. However, when for example an elderly person slips and falls down, the house might recognize that something is off. For example, the light in the bathroom stays on for a prolonged time. Algorithms in the cloud can use high-level data on the use of appliances and lights to detect if there is an anomaly with respect to normal activities in the house. In the thesis below deep learning (auto-encoders) are used to detect if something is out of the ordinary.
Crownstone wants to make your home smarter. To configure your home through your smartphone is a necessary evil. Since December 2018 Crownstone can be controlled from an Alexa device. Control is nice, but we want something more. It would be great to actually tell Crownstones to perform certain actions when you enter the room. This means that you can program or configure the Crownstones by speech. Moreover, if the AI is turning on a light, you might wonder: “Why are the lights turned on?”. This type of retrospection is yet another level of intelligence that would be awesome for our customers!
A first implementation can be found in the thesis below:
You are the boss of your own data. Crownstone wants to provide secure methods in which your data can be analysed in the cloud without running the risk that it is used for secondary purposes. This can be achieved by so-called homomorphic encryption. It is a form of encryption that allows computations on encrypted data without the need of decrypting it. How does it work? For example, Crownstone communicates in encrypted form the power consumption of every device in a household to the cloud. The cloud has no way to decrypt this data. Thanks to this type of homomorphic encryption the cloud can aggregate the consumption by a sum operator that works on the encrypted data. This sum is also in encrypted form. The cloud does not know the result either! The encrypted result is returned to the user. The user can decrypt the result and voila!
There is now third generation homomorphic encryption, but it is still very computationally expensive. To be really used in practice, a lot of theoretical advances have to be made. This technique has the potential to become more important than end-to-end encryption and blockchain combined. There is a homomorphic encryption consortium with members like Intel, Microsoft, IBM, and members who completely focus on this technology, such as Duality Technologies, Inpher, and Ixup.
There are other topics that we have studied in the past. For example distributed control of fridges for demand response on the grid. More can be found in the Crownstone hall of fame.