Smart Home 3D Webapp with three.js MQTT sensors and UWB

Project documentation page

source code github repo

Summary
See your 3d rooms model real time colored with your temperature sensor, click on the 3d light bulb model to switch the real light on, see the light switched on in the 3d model as well, see if windows are open by seeing an actual 3d window model open or closed,…
While Video Games have reached incredible realism, Smart Home applications still struggle with 2D apps. Each IoT device suggest its own isolated app.
This project’s idea is not to provide the best 3d web app, rather collects different pieces of frameworks enabling each of us to customize his 3d interactions with his own devices.
The main idea is to model items in blender (or use existing ones provided here) that have animations mapped to properties names. Thanks the the .glTF export and import in three.js, those items become interactive through mouse clicks and gets their properties updated from external events such as connections to mqtt and hue gateway.

how do u interface the home 3d with physical devices? is it through mqtt or web services?

As you see in this line, the webapp is using a javascript mqtt client that connects to the broker with a websocket

Here’s the link to the used dependency

that’s awesome thanks! do u know how or if it is possible to link or control a 3D Blender object with websocket using python or something?

yes, that’s the idea, you can basically connect any MQTT sensor value to any 3d parameter, rotation, position or color.
I’m using three.js which is one of the best libraries to use webgl from javascript

3D Blender objects are exported and imported through the glTF format, from there on, they’re like aby other webgl object for which you can control the 3D position and orientation.

The problem with blender design is that it’s not easy to be identifying objects and assigning them mouse controls or movements or color changes, in this webpage below, I describe a framework with which I interconnect objects designed inside the Blender scene, and how I assign them custom properties that help to automatically recognize them in the webapp and add interactivity without any custom js code.

for info, Hackaday wrote a post about this project, it’s a bit ahead of its time as 3d power is not yet that strong on smartphones, but merging 3d into the web and making its usage as easy as 2d web times is where the web is going

1 Like

I hav been trying to convince my brother to try Unity bc he is a graphic designer and I have interests in 3d modelings. The problem is that to integrate iot with Unity on a webapp seems like to take too much efforts n im not sure if it will work. I wonder how three.js performs on mobile devices? I would like to hav a 3d model of my house to position my sensors, and use it to create a heat map to better analyze different data.

is it possible to create a 3d house map using a flat map with data set describing dimensions of different elements with three.js?

Three.js is not a modeling tool, it’s a rendering engine. I do not think that you can have a descent model without using a modeling tool. Some tools like Sweet Home 3D, offer functions like you describe, super easy for those who are not familiar with professional CAD, you just draw lines and then become walls and place doors and windows easily. Now as soon as you hit something custom or special, you need to fallback on a real CAD tool.
I modeled my flat using Fusion360, because that’s for me the fastest way with parametric modeling, then the I exported it to blender because that’s the only tool that can generate a glTF which is a format that contains meta information of about all objects in the scene that can be reused with javascript once loaded.
If unity can load glTF, then you can use it, what I developed in this project is open source and you could adapt it. Three.js is open source, free and multi platform, and at least in this case the connection of IoT sensors through MQTT and it mapping to all properties colors, positions,… is already done.

great thanks. I will look into using it for home automation. as well as restaurant or retail store layouts. with sensors, i would b able to analyze the traffic at certain locations. it would b great to quickly generate a 3d map without having to create separate model each time.