Hey, thanks for having me today, really appreciate the invitation. Today I’d like to talk a little bit about a lockdown project of mine called „An Arduino meets GraphQL and the weather.“ But first things first: I am Sascha, I am a front end developer based in south of Wels an I like to build web services an exploring new stuff in my spare time. So, to begin with the why, what and how, what I will be talking about.
The main motivation was that I always wanted a weather station, and instead of just buying one I thought I might face the challenge to possibly build one on my own. And for that I defined 4 must have features: it should be remotely accessible - so no matter wherever I am I should have the possibility of looking up the latest temperature values. It should be self-hosted because I wanted to model my own data, I wanted to avoid any kind of subscription fee or subscription plan. Of course an API so that querying data and updating data is as easy as possible. And it should be as cheap as possible because I considered it as a hobby project and just didn’t want to invest a fortune.
So to sum it up in one sentence was the question I asked myself: What if I could hook up an Arduino to a GraphQL API and store live sensor data in a database?
So the next step was to form a sort of a theoretical approach. First I needed to get current time using the NTP protocol because an Arduino doesn’t have any local clock system attached to it. Then I need to calculate the minutes before the next interval. I wanted to collect then the sensor data, temperature values, humidity values, uploaded the results to the GraphQL API and then do it all over again in the next 30 minutes. The sleep phase is very important because it makes it independent to the weather station itself, whether it loses or regains power because then I can define a fixed interval based on the universal clock, the universal time zone to upload the data.
So where to start. I looked for an as cheap as possible development board and found one in the D1 mini. It is a small Arduino compatible development board and the main advantage of it is it has already Wi-Fi on board. So without attaching any other stuff to it, you can already connected it to the internet and perform communication tasks. Of course the Apollo GraphQL server for the API and everything web-based is hosted on Netlify and MongoDB Atlas because both offer generous free tiers. I'm not affiliated with any of those.
What I ended up with was of course the D1 mini, two different temperature sensors, one being the DS18B20 and one being the DHT11. The DS18B20 is quite cool because it's attached to a cable so you can place it a little bit further away than the base station. And the DHT11 also features humidity values to the temperature values. I also opted for attaching a micro SD module. So whenever the development board loses the connection to the Wi-Fi, the fetched temperature data can be stored to an SD card and therefore serves as a redundant memory. The total prize is about 12 to 15 euros - so I consider that as rather cheap if you buy lots of pieces for each of the sensors then you even get the price down significantly.
So this is more or less the first demo setup. I wired everything up – on the left hand side you can see the micro SD module. On the center right is the development board with the Wi-Fi chip. And on the top right there is this blueish DHT11 temperature sensor.
But all of the setup is mainly useless if you don't provide the correct firmware. So I needed to develop one and this is basically the next step. So for the development I chose to call this chapter “C++ and the Web” and if you are like me trying to avoid C++ at all costs because of bad memories made during your university, the main disadvantage is you can't really get away with anything else because Arduino strongly depends on a precompiled C++ firmware. Unlike raspberry PIs for example because they feature a 32- to 64-bit ARM processor and therefore are capable of running a fully fledged Linux operating system. Arduinos consist of a small 8-bit microcontroller with a flash base of around 4 to 8 MB and therefore you have to write the whole instructions of what the Arduino has to do on your own using the Arduino IDE. And the whole ecosystem is quite cool because it offers a multitude of freely available libraries. For every sensor, for every http-project you're trying to create the Arduino ecosystem provides a library and you can just hook it up altogether in your source code and basically you are ready to go within a matter of minutes. The firmware you have to write mainly consists of mandatory setup and loop function. So you have to imagine if you start up the Arduino then it first calls the setup function and sets up the connection to the Wi-Fi, sets the IP address configuration, initiates the NTP client for example and then goes on to call the loop function over and over and over again until it loses power. The IDE itself is basically a one-stop shop for everything because it already contains drivers and the compiler. So you open up a new project on the IDE, you possibly even just load a predefined demo code, you click on a button, it gets validated and compiled. And if the Arduino is hooked up to your computer using a USB cable then it gets flashed and everything runs from there. So pretty smooth.
Again back to the theoretical approach. If you look at the flow chart it's basically the perfect tailormade flow for the loop function because it is a single-threaded loop function. I call the synchronization to the NTP-client, I let the whole process wait for a certain amount of time until they collect the sensor data and then upload the results again to the GraphQL and wait for another 30 minutes. So basically, that's that and that's all the weather station is doing for the whole day and for a whole night as long as it's running.
But what about GraphQL though? I mentioned it shortly at the beginning but didn't talk about it during the last few slides. So the GraphQL communication was a bit of another challenge because if you take a look at those screenshots the top one is a curl request to a GraphQL API, which queries Star Wars film data. In this case the properties for the title, the episode ID and the release date. And that all happens within your terminal but it is only a syntactic sugar to what's pictured below because http works as a single string that is split by carriage return escape characters and if you take a look at the post at the first line we create a post request to the root path of the host. We set the content type to application JSON and set the content length to 64 bytes because the body will be 64 bytes long and the server will check whether the whole body was transmitted or not. Then we need two carriage return escape characters so that the server knows that the headers are now complete and the body is now being transmitted. So we have the GraphQL body just as within the curl request above and then with the connection close, we close the connection and basically that’s what happens if you try to post a GraphQL query to a GraphQL server. Since you probably know by now Arduinos are a bit low level – I knew I had to align with the bottom screenshot and had to stuff everything together in a string and so basically for the weather station it looks like this. The first one is a string value variable called body with a lot of backslash escape characters for the string. And then we have a lot of string concatenations it takes the preloaded or that just fetched temperature data, appends it to the string, then calculates the length of the body and then appends it to the request below. So the request below is basically just the same as we have just seen in the previous slide. We have the post requests at the first line, then we have at the end of it a carriage return escape character, we have to host, we have content tab, we have the content length, we have two carriage return escape characters. Then we append the body and then we close the connection and using client.print. Client is a variable where the Arduino has instantiated the http client, we will send the request to the GraphQL server. With the last function, the serial.printline, we can hook up the Arduino to some sort of debugging functionality.
So if you wire the Arduino via USB cable to your Arduino IDE you can read out the serial messages because otherwise you don't have any control what's going on right now.
So that's that for the development - now comes the final assembly, also called “Solder till your fingers burn” because I had to do a lot of soldiering and it's just not what I'm doing on a daily basis. So those are the impressions of the final soldiering. On the right hand side you see how it all came together, the resistors, the pins, also the micro SD shield. And below is a weather station without a micro SD shield. That's possible because the firmware is quite configurable, so you can select where the temperature sensors are attached, which temperature sensors are attached and whether or not a micro SD card is attached as well.
I then also built a website for displaying the data that has been gathered from the weather station and basically the whole source code is made open source and can be looked up at my GitHub repository. You will find the firmware source code in there, the website source code and the GraphQL server source code. Just to give you a short overview of what has been recorded so far this year. The weather station is online since somewhere between Christmas and New Year, last week we have had the highest temperature of 33 degrees on June 18th. Lowest temperature this year was minus 12.9 on February 13th. And so far more than 16.600 entries have been uploaded to the database from two individual weather stations.
So that's that. Thank you very much! If you want to look up anything the source code is online available and if you need further help, just shoot me a question on Twitter. Thanks!