Azure IoT Hub MQTT LoRa Field Gateway

Back in April I started working on an MQTT LoRa Field gateway which was going to support a selection of different Software as a service(SaaS) Internet of Things IoT) platforms.

I now have a working Azure IoT Hub plug-in (Azure IoT Central support as planned as well) with the first iteration focused on Device to Cloud (D2C) messaging. In a future iteration I will add Cloud to Device messaging(C2D).

My applications use a lightweight, easy to implemented protocol which is intended for hobbyist and educational use rather than commercial applications (I have been working on a more secure version as yet another side project)

I have a number of sample Arduino with Dragino LoRa Shield for Arduino, MakerFabs Maduino, Dragino LoRa Mini Dev, M2M Low power Node and Netduino with Elecrow LoRa RFM95 Shield etc. clients. These work with both my platform specific (Adafruit.IO, Azure IoT Hub/Central) gateways and protocol specific field gateways.

Azure IoT Hub Device Explorer Data Display

When the application is first started it creates a minimal configuration file which should be downloaded, the missing information filled out, then uploaded using the File explorer in the Windows device portal.

{
  "MQTTUserName": "YourIoTHubHub.azure-devices.net/MQTTLoRa915MHz/api-version=2018-06-30",
  "MQTTPassword": "SharedAccessSignature sr=YourIoTHubHub.azure-devices.net%2Fdevices%2FMQTTLoRa915MHz&sig=123456789012345678901234567890123456789012345%3D&se=1574673583",
  "MQTTClientID": "MQTTLoRa915MHz",
  "MQTTServer": "YourIoTHubHub.azure-devices.net",
  "Address": "LoRaIoT1",
  "Frequency": 915000000.0,
  "MessageHandlerAssembly": "Mqtt.IoTCore.FieldGateway.LoRa.AzureIoTHub",
  "PlatformSpecificConfiguration": ""
}

The application logs debugging information to the Windows 10 IoT Core ETW logging Microsoft-Windows-Diagnostics-LoggingChannel

MQTT LoRa Gateway with Azure IoT Hub plug-in

The message handler uploads all values in an inbound messages in one MQTT message.

namespace devMobile.Mqtt.IoTCore.FieldGateway
{
   using System;
   using System.Diagnostics;
   using System.Text;
   using Windows.Foundation.Diagnostics;

   using devMobile.IoT.Rfm9x;
   using MQTTnet;
   using MQTTnet.Client;
   using Newtonsoft.Json.Linq;
   using Newtonsoft.Json;

   public class MessageHandler : IMessageHandler
   {
      private LoggingChannel Logging { get; set; }
      private IMqttClient MqttClient { get; set; }
      private Rfm9XDevice Rfm9XDevice { get; set; }
      private string PlatformSpecificConfiguration { get; set; }

      void IMessageHandler.Initialise(LoggingChannel logging, IMqttClient mqttClient, Rfm9XDevice rfm9XDevice, string platformSpecificConfiguration)
      {
         LoggingFields processInitialiseLoggingFields = new LoggingFields();

         this.Logging = logging;
         this.MqttClient = mqttClient;
         this.Rfm9XDevice = rfm9XDevice;
         this.PlatformSpecificConfiguration = platformSpecificConfiguration;
      }

      async void IMessageHandler.Rfm9XOnReceive(Rfm9XDevice.OnDataReceivedEventArgs e)
      {
         LoggingFields processReceiveLoggingFields = new LoggingFields();
         char[] sensorReadingSeparators = { ',' };
         char[] sensorIdAndValueSeparators = { ' ' };

         processReceiveLoggingFields.AddString("PacketSNR", e.PacketSnr.ToString("F1"));
         processReceiveLoggingFields.AddInt32("PacketRSSI", e.PacketRssi);
         processReceiveLoggingFields.AddInt32("RSSI", e.Rssi);

         string addressBcdText = BitConverter.ToString(e.Address);
         processReceiveLoggingFields.AddInt32("DeviceAddressLength", e.Address.Length);
         processReceiveLoggingFields.AddString("DeviceAddressBCD", addressBcdText);

         string messageText;
         try
         {
            messageText = UTF8Encoding.UTF8.GetString(e.Data);
            processReceiveLoggingFields.AddString("MessageText", messageText);
         }
         catch (Exception ex)
         {
            processReceiveLoggingFields.AddString("Exception", ex.ToString());
            this.Logging.LogEvent("PayloadProcess failure converting payload to text", processReceiveLoggingFields, LoggingLevel.Warning);
            return;
         }

         // Chop up the CSV text
         string[] sensorReadings = messageText.Split(sensorReadingSeparators, StringSplitOptions.RemoveEmptyEntries);
         if (sensorReadings.Length < 1)
         {
            this.Logging.LogEvent("PayloadProcess payload contains no sensor readings", processReceiveLoggingFields, LoggingLevel.Warning);
            return;
         }

         JObject payloadJObject = new JObject();

         JObject feeds = new JObject();

         // Chop up each sensor read into an ID & value
         foreach (string sensorReading in sensorReadings)
         {
            string[] sensorIdAndValue = sensorReading.Split(sensorIdAndValueSeparators, StringSplitOptions.RemoveEmptyEntries);

            // Check that there is an id & value
            if (sensorIdAndValue.Length != 2)
            {
               this.Logging.LogEvent("PayloadProcess payload invalid format", processReceiveLoggingFields, LoggingLevel.Warning);
               return;
            }

            string sensorId = string.Concat(addressBcdText, sensorIdAndValue[0]);
            string value = sensorIdAndValue[1];

            feeds.Add(sensorId.ToLower(), value);
         }
         payloadJObject.Add("feeds", feeds);

         string topic = $"devices/{MqttClient.Options.ClientId}/messages/events/";

         try
         {
            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topic)
               .WithPayload(JsonConvert.SerializeObject(payloadJObject))
               .WithAtLeastOnceQoS()
               .Build();
            Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync start", DateTime.UtcNow);
            await MqttClient.PublishAsync(message);
            Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync finish", DateTime.UtcNow);

            this.Logging.LogEvent("PublishAsync Azure IoTHub payload", processReceiveLoggingFields, LoggingLevel.Information);
         }
         catch (Exception ex)
         {
            processReceiveLoggingFields.AddString("Exception", ex.ToString());
            this.Logging.LogEvent("PublishAsync Azure IoTHub payload", processReceiveLoggingFields, LoggingLevel.Error);
         }
      }

      void IMessageHandler.MqttApplicationMessageReceived(MqttApplicationMessageReceivedEventArgs e)
      {
         LoggingFields processReceiveLoggingFields = new LoggingFields();

         processReceiveLoggingFields.AddString("ClientId", e.ClientId);
#if DEBUG
         processReceiveLoggingFields.AddString("Payload", e.ApplicationMessage.ConvertPayloadToString());
#endif
         processReceiveLoggingFields.AddString("QualityOfServiceLevel", e.ApplicationMessage.QualityOfServiceLevel.ToString());
         processReceiveLoggingFields.AddBoolean("Retain", e.ApplicationMessage.Retain);
         processReceiveLoggingFields.AddString("Topic", e.ApplicationMessage.Topic);

         this.Logging.LogEvent("MqttApplicationMessageReceived topic not processed", processReceiveLoggingFields, LoggingLevel.Error);
      }

      void IMessageHandler.Rfm9xOnTransmit(Rfm9XDevice.OnDataTransmitedEventArgs e)
      {
      }
   }
}

The formatting of the username and generation of password are password are a bit awkward and will be fixed in a future refactoring. Along with regenerating the SAS connection token just before it is due to expire.

Azure IoT Hub SAS Tokens revisited again

This post has been edited (2019-11-24) my original assumption about how DateTime.Kind unspecified was handled were incorrect.

As I was testing my Azure MQTT Test Client I noticed some oddness with MQTT connection timeouts and this got me wondering about token expiry times. So, I went searching again and found this Azure IoT Hub specific sample code

public static string generateSasToken(string resourceUri, string key, string policyName, int expiryInSeconds = 3600)
{
    TimeSpan fromEpochStart = DateTime.UtcNow - new DateTime(1970, 1, 1);
    string expiry = Convert.ToString((int)fromEpochStart.TotalSeconds + expiryInSeconds);

    string stringToSign = WebUtility.UrlEncode(resourceUri) + "\n" + expiry;

    HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(key));
    string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));

    string token = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", WebUtility.UrlEncode(resourceUri), WebUtility.UrlEncode(signature), expiry);

    if (!String.IsNullOrEmpty(policyName))
    {
        token += "&skn=" + policyName;
    }

    return token;
}

This code worked first time and was more flexible than mine which was a bonus. Though while running my MQTTNet based client I noticed the connection would drop after approximately 10mins (EDIT this was probably an unrelated networking issue).

A long time ago (25 years) I had issues sharing a Unix time value between an applications written with Borland C and Microsoft Visual C which made me wonder about Unix epoch base offsets.

So to test my theory I built a Unix epoch test harness console application

using System;

namespace UnixEpocTest
{
   class Program
   {
      static void Main(string[] args)
      {
         TimeSpan ttl = new TimeSpan(0, 0, 0);

         Console.WriteLine("Current time");
         Console.WriteLine($"Local     {DateTime.Now} {DateTime.Now.Kind}");
         Console.WriteLine($"UTC       {DateTime.UtcNow} {DateTime.UtcNow.Kind}");
         Console.WriteLine($"Unix DIY  {new DateTime(1970, 1, 1)} {new DateTime(1970, 1, 1).Kind}");
         Console.WriteLine($"Unix DIY+ {new DateTime(1970, 1, 1).ToUniversalTime()} {new DateTime(1970, 1, 1).ToUniversalTime().Kind}");
         Console.WriteLine($"Unix DIY  {new DateTime(1970, 1, 1, 0,0,0, DateTimeKind.Utc)}");
         Console.WriteLine($"Unix      {DateTime.UnixEpoch} {DateTime.UnixEpoch.Kind}");
         Console.WriteLine();

         TimeSpan fromEpochStart = DateTime.UtcNow - new DateTime(1970, 1, 1);
         TimeSpan fromEpochStartUtc = DateTime.UtcNow - new DateTime(1970, 1, 1,0,0,0, DateTimeKind.Utc);
         TimeSpan fromEpochStartUnixEpoch = DateTime.UtcNow - DateTime.UnixEpoch;

         Console.WriteLine("Epoch comparison");
         Console.WriteLine($"Local {fromEpochStart} {fromEpochStart.TotalSeconds.ToString("f0")} sec");
         Console.WriteLine($"UTC   {fromEpochStartUtc} {fromEpochStartUtc.TotalSeconds.ToString("f0")} sec");
         Console.WriteLine($"Epoc  {fromEpochStartUnixEpoch} {fromEpochStartUnixEpoch.TotalSeconds.ToString("f0")} sec");
         Console.WriteLine();

         TimeSpan afterEpoch = DateTime.UtcNow.Add(ttl) - new DateTime(1970, 1, 1);
         TimeSpan afterEpochUtC = DateTime.UtcNow.Add(ttl) - new DateTime(1970, 1, 1).ToUniversalTime();
         TimeSpan afterEpochEpoch = DateTime.UtcNow.Add(ttl) - DateTime.UnixEpoch;

         Console.WriteLine("Epoch calculation");
         Console.WriteLine($"Local {afterEpoch}");
         Console.WriteLine($"UTC   {afterEpochUtC}");
         Console.WriteLine($"Epoch {afterEpochEpoch}");
         Console.WriteLine();

         Console.WriteLine("Epoch DateTime");
         Console.WriteLine($"Local :{new DateTime(1970, 1, 1)}");
         Console.WriteLine($"UTC   :{ new DateTime(1970, 1, 1).ToUniversalTime()}");

         Console.WriteLine("Press ENTER to exit");
         Console.ReadLine();

         Console.WriteLine("Hello World!");
      }
   }
}

EDIT: I now think the UtcNow to “unspecified” kind mathematics was being handled correctly. I have updated the code to use the DateTime.UnixEpoch constant so the code is more readable.

public static string generateSasToken(string resourceUri, string key, string policyName, int expiryInSeconds = 900)
      {
         TimeSpan fromEpochStart = DateTime.UtcNow - DateTime.UnixEpoch;
         string expiry = Convert.ToString((int)fromEpochStart.TotalSeconds + expiryInSeconds);

         string stringToSign = WebUtility.UrlEncode(resourceUri) + "\n" + expiry;

         HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(key));
         string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));

         string token = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", WebUtility.UrlEncode(resourceUri), WebUtility.UrlEncode(signature), expiry);

         if (!String.IsNullOrEmpty(policyName))
         {
            token += "&skn=" + policyName;
         }

         return token;
      }

I need to test the expiry of my SAS Tokens some more especially with the client running on my development machine (NZT which is currently UTC+13) and in Azure (UTC timezone)

Azure IoT Hub SAS Tokens revisited

A long time ago I wrote a post about uploading telemetry data to an Azure Event Hub from a Netduino 3 Wifi using HTTPS. To send messages to the EventHub I had to create a valid SAS Token which took a surprising amount of effort because of the reduced text encoding/decoding and cryptographic functionality available in .NET Micro Framework v4.3 (NetMF)

// Create a SAS token for a specified scope. SAS tokens are described in http://msdn.microsoft.com/en-us/library/windowsazure/dn170477.aspx.
private static string CreateSasToken(string uri, string keyName, string key)
{
   // Set token lifetime to 20 minutes. When supplying a device with a token, you might want to use a longer expiration time.
   uint tokenExpirationTime = GetExpiry(20 * 60);
 
   string stringToSign = HttpUtility.UrlEncode(uri) + "\n" + tokenExpirationTime;
 
   var hmac = SHA.computeHMAC_SHA256(Encoding.UTF8.GetBytes(key), Encoding.UTF8.GetBytes(stringToSign));
   string signature = Convert.ToBase64String(hmac);
 
   signature = Base64NetMf42ToRfc4648(signature);
 
   string token = "SharedAccessSignature sr=" + HttpUtility.UrlEncode(uri) + "&sig=" + HttpUtility.UrlEncode(signature) + "&se=" + tokenExpirationTime.ToString() + "&skn=" + keyName;
 
   return token;
}
 
private static string Base64NetMf42ToRfc4648(string base64netMf)
{
   var base64Rfc = string.Empty;
 
   for (var i = 0; i < base64netMf.Length; i++)
   {
      if (base64netMf[i] == '!')
      {
         base64Rfc += '+';
      }
      else if (base64netMf[i] == '*')
      {
         base64Rfc += '/';
      }
      else
      {
         base64Rfc += base64netMf[i];
      }
   }
   return base64Rfc;
}
 
static uint GetExpiry(uint tokenLifetimeInSeconds)
{
   const long ticksPerSecond = 1000000000 / 100; // 1 tick = 100 nano seconds
 
   DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0);
   TimeSpan diff = DateTime.Now.ToUniversalTime() - origin;
 
   return ((uint)(diff.Ticks / ticksPerSecond)) + tokenLifetimeInSeconds;
}

Initially for testing my Azure MQTT Test Client I manually generated the SAS tokens using Azure Device Explorer but figured it would be better if the application generated them.

An initial search lead to this article about how to generate a SAS token for an Azure Event Hub in multiple languages. For my first attempt I “copied and paste” the code sample for C# (I also wasn’t certain what to put in the KeyName parameter) and it didn’t work.

private static string createToken(string resourceUri, string keyName, string key)
{
    TimeSpan sinceEpoch = DateTime.UtcNow - new DateTime(1970, 1, 1);
    var week = 60 * 60 * 24 * 7;
    var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + week);
    string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
    HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
    var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
    var sasToken = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}&skn={3}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry, keyName);
    return sasToken;
}

By comparing the Device Explorer and C# generated SAS keys I worked out the keyName parameter was unnecessary so I removed.

private static string createToken(string resourceUri, string key)
{
    TimeSpan sinceEpoch = DateTime.UtcNow - new DateTime(1970, 1, 1);
    var week = 60 * 60 * 24 * 7;
    var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + week);
    string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
    HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
    var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
    var sasToken = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry);
    return sasToken;
}

The shared SAS token now looked closer to what I was expecting but the MQTTNet ConnectAsync was failing with an authentication exception. After looking at the Device Explorer SAS Key code, my .NetMF implementation and the code for the IoT Hub SDK I noticed the encoding for the HMAC Key was different. Encoding.UTF8.GetBytes vs. Convert.FromBase64String.

 private static string createToken(string resourceUri,string key, TimeSpan ttl)
      {
         TimeSpan afterEpoch = DateTime.UtcNow.Add( ttl ) - new DateTime(1970, 1, 1);

         string expiry = afterEpoch.TotalSeconds.ToString("F0");
         string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
         HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(key));
         string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
         return  String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry);
      }

This approach appears to work reliably in my test harness.

MQTTnet client with new SAS Key Generator

User beware DIY Crypto often ends badly

Azure IoT Hub with MQTTnet

As I’m testing my Message Queue Telemetry Transport(MQTT) LoRa gateway I’m building a proof of concept(PoC) .Net core console application for each IoT platform I would like to support.

This PoC was to confirm that my device could connect to the Microsoft Azure IoT Hub MQTT API then format topics and payloads correctly.

Azure IoT Hub MQTT Console Client

I had tried with a couple of different MQTT libraries from micro controllers and embedded devices without success. With the benefit of hindsight (plus this article) I think I had the SAS key format wrong.

The Azure IoT Hub MQTT broker requires only a server name (fully resolved CName), device ID and SAS Key.

   class Program
   {
      private static IMqttClient mqttClient = null;
      private static IMqttClientOptions mqttOptions = null;
      private static string server;
      private static string username;
      private static string password;
      private static string clientId;
      private static string topicD2C;
      private static string topicC2D;

      static void Main(string[] args)
      {
         MqttFactory factory = new MqttFactory();
         mqttClient = factory.CreateMqttClient();

         if (args.Length != 3)
         {
            Console.WriteLine("[AzureIoTHubHostName] [deviceID] [SASKey]");
            Console.WriteLine("Press <enter> to exit");
            Console.ReadLine();
            return;
         }

         server = args[0];
         clientId = args[1];
         sasKey= args[2];

         username = $"{server}/{clientId}/api-version=2018-06-30";
         topicD2C = $"devices/{clientId}/messages/events/";
         topicC2D = $"devices/{clientId}/messages/devicebound/#";

         Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId}");

         mqttOptions = new MqttClientOptionsBuilder()
            .WithTcpServer(server, 8883)
            .WithCredentials(username, sasKey)
            .WithClientId(clientId)
            .WithTls()
            .Build();

         mqttClient.UseDisconnectedHandler(new MqttClientDisconnectedHandlerDelegate(e => MqttClient_Disconnected(e)));
         mqttClient.UseApplicationMessageReceivedHandler(new MqttApplicationMessageReceivedHandlerDelegate(e => MqttClient_ApplicationMessageReceived(e)));
         mqttClient.ConnectAsync(mqttOptions).Wait();

         mqttClient.SubscribeAsync(topicC2D, MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce).GetAwaiter().GetResult();

         while (true)
         {
            JObject payloadJObject = new JObject();

            payloadJObject.Add("OfficeTemperature", "22." + DateTime.UtcNow.Millisecond.ToString());
            payloadJObject.Add("OfficeHumidity", (DateTime.UtcNow.Second + 40).ToString());

            string payload = JsonConvert.SerializeObject(payloadJObject);
            Console.WriteLine($"Topic:{topicD2C} Payload:{payload}");

            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topicD2C)
               .WithPayload(payload)
               .WithAtLeastOnceQoS()
            .Build();

            Console.WriteLine("PublishAsync start");
            mqttClient.PublishAsync(message).Wait();
            Console.WriteLine("PublishAsync finish");

            Thread.Sleep(30100);
         }
      }

      private static void MqttClient_ApplicationMessageReceived(MqttApplicationMessageReceivedEventArgs e)
      {
         Console.WriteLine($"ClientId:{e.ClientId} Topic:{e.ApplicationMessage.Topic} Payload:{e.ApplicationMessage.ConvertPayloadToString()}");
      }

      private static async void MqttClient_Disconnected(MqttClientDisconnectedEventArgs e)
      {
         Debug.WriteLine("Disconnected");
         await Task.Delay(TimeSpan.FromSeconds(5));

         try
         {
            await mqttClient.ConnectAsync(mqttOptions);
         }
         catch (Exception ex)
         {
            Debug.WriteLine("Reconnect failed {0}", ex.Message);
         }
      }
   }

Overall the initial configuration went smoothly after I figured out the required Quality of Service (QoS) settings, and the SAS Key format.

Using the approach described in the Microsoft documentation I manually generated the SAS Key.(In my Netduino samples I have code for generating a SAS Key in my HTTPS Azure IoT Hub Client)

Azure Device Explorer Device Management
Azure Device Explorer SAS Key Generator

Once I had the configuration correct I could see telemetry from the device and send it messages.

Azure Device Explorer Data View

In a future post I will upload data to the Azure IoT Central for display. Then explore using a “module” attached to a device which maybe useful for my field gateway.

Thingspeak with MQTTnet

As I’m testing my Message Queue Telemetry Transport(MQTT) LoRa gateway I’m building a proof of concept(PoC) .Net core console application for each IoT platform I would like to support.

This PoC was to confirm that I could connect to the Thingspeak MQTT API then format topics and payloads correctly.

MQTT Console Client

The Thingspeak MQTT broker, username, password, and client ID, Channel, writeAPIKey, ClientID, channelData, are the minimum command line options required. The channelSubscribe, readAPIkey and field are optional.

   class Program
   {
      private static IMqttClient mqttClient = null;
      private static IMqttClientOptions mqttOptions = null;
      private static string server;
      private static string username;
      private static string password;
      private static string writeApiKey;
      private static string clientId;
      private static string channelData;
      private static string channelSubscribe;
      private static string readApiKey;
      private static string field;

      static void Main(string[] args)
      {
         MqttFactory factory = new MqttFactory();
         mqttClient = factory.CreateMqttClient();

         if ((args.Length != 6) && (args.Length != 8) && (args.Length != 9))
         {
            Console.WriteLine("[MQTT Server] [UserName] [Password] [WriteAPIKey] [ClientID] [Channel]");
            Console.WriteLine("[MQTT Server] [UserName] [Password] [WriteAPIKey] [ClientID] [Channel] [channelSubscribe] [ReadApiKey]");
            Console.WriteLine("[MQTT Server] [UserName] [Password] [WriteAPIKey] [ClientID] [Channel] [channelSubscribe] [ReadApiKey] [Field]");
            Console.WriteLine("Press <enter> to exit");
            Console.ReadLine();
            return;
         }

         server = args[0];
         username = args[1];
         password = args[2];
         writeApiKey = args[3];
         clientId = args[4];
         channelData = args[5];

         if (args.Length == 6)
         {
            Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} ChannelData:{channelData}");
         }

         if (args.Length == 8)
         {
            channelSubscribe = args[6];
            readApiKey = args[7];
            Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} ChannelData:{channelData} ChannelSubscribe:{channelSubscribe}");
         }

         if (args.Length == 9)
         {
            channelSubscribe = args[6];
            readApiKey = args[7];
            field = args[8];
            Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} ChannelData:{channelData} ChannelSubscribe:{channelSubscribe} Field:{field}");
         }

         mqttOptions = new MqttClientOptionsBuilder()
            .WithTcpServer(server)
            .WithCredentials(username, password)
            .WithClientId(clientId)
            .WithTls()
            .Build();

         mqttClient.ConnectAsync(mqttOptions).Wait();

         if (args.Length == 8)
         {
            string topic = $"channels/{channelSubscribe}/subscribe/fields/+/{readApiKey}";
            
            Console.WriteLine($"Subscribe Topic:{topic}");

            mqttClient.SubscribeAsync(topic, MQTTnet.Protocol.MqttQualityOfServiceLevel.AtMostOnce).Wait(); 
            mqttClient.UseApplicationMessageReceivedHandler(new MqttApplicationMessageReceivedHandlerDelegate(e => MqttClient_ApplicationMessageReceived(e)));
         }
         if (args.Length == 9)
         {
            string topic = $"channels/{channelSubscribe}/subscribe/fields/{field}/{readApiKey}";

            Console.WriteLine($"Subscribe Topic:{topic}");

            mqttClient.SubscribeAsync(topic, MQTTnet.Protocol.MqttQualityOfServiceLevel.AtMostOnce).Wait();
            mqttClient.UseApplicationMessageReceivedHandler(new MqttApplicationMessageReceivedHandlerDelegate(e => MqttClient_ApplicationMessageReceived(e)));
         }

         mqttClient.UseDisconnectedHandler(new MqttClientDisconnectedHandlerDelegate(e => MqttClient_Disconnected(e)));

         string topicTemperatureData = $"channels/{channelData}/publish/{writeApiKey}";

         Console.WriteLine();

         while (true)
         {
            string value = "field1=22." + DateTime.UtcNow.Millisecond.ToString() + "&field2=60." + DateTime.UtcNow.Millisecond.ToString();
            Console.WriteLine($"Publish Topic {topicTemperatureData}  Value {value}");

            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topicTemperatureData)
               .WithPayload(value)
               .WithQualityOfServiceLevel(MQTTnet.Protocol.MqttQualityOfServiceLevel.AtMostOnce)
//               .WithRetainFlag()
            .Build();

            Console.WriteLine("PublishAsync start");
            mqttClient.PublishAsync(message).Wait();
            Console.WriteLine("PublishAsync finish");
            Console.WriteLine();

            Thread.Sleep(30100);
         }
      }

      private static void MqttClient_ApplicationMessageReceived(MqttApplicationMessageReceivedEventArgs e)
      {
         Console.WriteLine($"ApplicationMessageReceived ClientId:{e.ClientId} Topic:{e.ApplicationMessage.Topic} Qos:{e.ApplicationMessage.QualityOfServiceLevel} Payload:{e.ApplicationMessage.ConvertPayloadToString()}");
         Console.WriteLine();
      }

      private static async void MqttClient_Disconnected(MqttClientDisconnectedEventArgs e)
      {
         Debug.WriteLine("Disconnected");
         await Task.Delay(TimeSpan.FromSeconds(5));

         try
         {
            await mqttClient.ConnectAsync(mqttOptions);
         }
         catch (Exception ex)
         {
            Debug.WriteLine("Reconnect failed {0}", ex.Message);
         }
      }
   }
Channel configuration

The ThingSpeak channel setup has many attributes (link to an external site, link to a GitHub repository, fields for elevation, latitude, longitude etc. ) but only 8 data fields which seems a bit limiting (some of my sensor nodes report more than 8 values).

Dashboard

The Thingspeak dashboard configuration was fairly simple, though the maximum of eight sequentially numbered of inputs (Fields) might require some mapping code.

Overall the initial configuration went smoothly after I figured out the required Quality of Service (QoS) settings, retain flag usage and the different APIKeys on the publish vs. subscribe topics.

I have not explored the advanced analysis enabled by the tight integration with MATLAB which could be quite an advantage for applications requiring that sort of functionality.

ubidots MQTT LoRa Field Gateway

Back in April I started working on an MQTT LoRa Field gateway which was going to support a selection of different Software as a service(SaaS) Internet of Things(IoT) platforms.

After a long pause in development I have a working ubidots client and have 3 proof of concept (PoC) integrations for Adafruit.IO, AskSensors, and Losant. I am also working on Azure IoT Hub, Azure IoT Central. The first iteration is focused on Device to Cloud (D2C) messaging in the next iteration I will add Cloud to Device where viable(C2D).

My applications use a lightweight, easy to implemented protocol which is intended for hobbyist and educational use rather than commercial applications (I have been working on a more secure version as yet another side project)

I have a number of sample Arduino with Dragino LoRa Shield for Arduino, MakerFabs Maduino, Dragino LoRa Mini Dev, M2M Low power Node and Netduino with Elecrow LoRa RFM95 Shield etc. clients. These work with both my platform specific (Adafruit.IO, Azure IoT Central) gateways and protocol specific field gateways.

Ubidots dashboard

When the application is first started it creates a minimal configuration file which should be downloaded, the missing information filled out, then uploaded using the File explorer in the Windows device portal.

{
  "MQTTUserName": "Ubidots generated usname here",
  "MQTTPassword": "NotVerySecure",
  "MQTTClientID": "MQTTLoRaGateway",
  "MQTTServer": "industrial.api.ubidots.com",
  "Address": "LoRaIoT1",
  "Frequency": 915000000.0,
  "MessageHandlerAssembly": "Mqtt.IoTCore.FieldGateway.LoRa.Ubidots",
  "PlatformSpecificConfiguration": ""
}

The application logs debugging information to the Windows 10 IoT Core ETW logging Microsoft-Windows-Diagnostics-LoggingChannel

MQTT LoRa Field Gateway with ubidots plugin generated telemetry
ubidots device management
ubidot managment

The message handler uploads all values in an inbound messages in one MQTT message using the ubidots MQTT message format

async void IMessageHandler.Rfm9XOnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e)
{
	LoggingFields processReceiveLoggingFields = new LoggingFields();
	JObject telemetryDataPoint = new JObject();
	char[] sensorReadingSeparators = { ',' };
	char[] sensorIdAndValueSeparators = { ' ' };

	processReceiveLoggingFields.AddString("PacketSNR", e.PacketSnr.ToString("F1"));
	processReceiveLoggingFields.AddInt32("PacketRSSI", e.PacketRssi);
	processReceiveLoggingFields.AddInt32("RSSI", e.Rssi);

	string addressBcdText = BitConverter.ToString(e.Address);
	processReceiveLoggingFields.AddInt32("DeviceAddressLength", e.Address.Length);
	processReceiveLoggingFields.AddString("DeviceAddressBCD", addressBcdText);

	string messageText;
	try
	{
		messageText = UTF8Encoding.UTF8.GetString(e.Data);
		processReceiveLoggingFields.AddString("MessageText", messageText);
	}
	catch (Exception ex)
	{
		processReceiveLoggingFields.AddString("Exception", ex.ToString());
		this.Logging.LogEvent("PayloadProcess failure converting payload to text", processReceiveLoggingFields, LoggingLevel.Warning);
		return;
	}

	// Chop up the CSV text
	string[] sensorReadings = messageText.Split(sensorReadingSeparators, StringSplitOptions.RemoveEmptyEntries);
	if (sensorReadings.Length < 1)
	{
		this.Logging.LogEvent("PayloadProcess payload contains no sensor readings", processReceiveLoggingFields, LoggingLevel.Warning);
		return;
	}

	// Chop up each sensor read into an ID & value
	foreach (string sensorReading in sensorReadings)
	{
		string[] sensorIdAndValue = sensorReading.Split(sensorIdAndValueSeparators, StringSplitOptions.RemoveEmptyEntries);
		// Check that there is an id & value
		if (sensorIdAndValue.Length != 2)
		{
			this.Logging.LogEvent("PayloadProcess payload invalid format", processReceiveLoggingFields, LoggingLevel.Warning);
			return;
		}

		string sensorId = sensorIdAndValue[0];
		string value = sensorIdAndValue[1];

		telemetryDataPoint.Add(addressBcdText + sensorId, Convert.ToDouble(value));
	}
	processReceiveLoggingFields.AddString("MQTTClientId", MqttClient.Options.ClientId);

	string stateTopic = string.Format(stateTopicFormat, MqttClient.Options.ClientId);

	try
	{
		var message = new MqttApplicationMessageBuilder()
			.WithTopic(stateTopic)
			.WithPayload(JsonConvert.SerializeObject(telemetryDataPoint))
			.WithAtLeastOnceQoS()
			.Build();
		Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync start", DateTime.UtcNow);
		await MqttClient.PublishAsync(message);
		Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync finish", DateTime.UtcNow);

		this.Logging.LogEvent("PublishAsync Ubidots payload", processReceiveLoggingFields, LoggingLevel.Information);
	}
	catch (Exception ex)
	{
		processReceiveLoggingFields.AddString("Exception", ex.ToString());
		this.Logging.LogEvent("PublishAsync Ubidots payload", processReceiveLoggingFields, LoggingLevel.Error);
	}
}

The “automagic” provisioning of feeds does make setting up small scale systems easier, though I’m not certain how well it would scale.

Some of the fields weren’t obviously editable e.g.”ÄPI Label” in device configuration which I only discovered by clicking on them..

The limitations of the free account meant I couldn’t evaluate ubidots in much depth but what was available appeared to be robust and reliable (Nov 2019).

Adafruit.IO MQTT LoRa Field Gateway

Back in April I started working on an MQTT LoRa Field gateway which was going to support a selection of different Software as a service(SaaS) Internet of Things(IoT) platforms.

After a long pause in development I have a working AdaFruit.IO client and have 3 proof of concept (PoC) integrations for AskSensors, Losant and Ubidots. I am also working on Azure IoT Hub, Azure IoT Central clients. The first iteration is focused on Device to Cloud (D2C) messaging in the next iteration I will add Cloud to Device where viable(C2D).

My applications use a lightweight, easy to implemented protocol which is intended for hobbyist and educational use rather than commercial applications (I have been working on a more secure version as yet another side project)

I have a number of sample Arduino with Dragino LoRa Shield for Arduino, MakerFabs Maduino, Dragino LoRa Mini Dev, M2M Low power Node and Netduino with Elecrow LoRa RFM95 Shield etc. clients. These work with both my platform specific (Adafruit.IO, Azure IoT Central) gateways and protocol specific field gateways.

Maduino client dashboard

When the application is first started it creates a minimal configuration file which should be downloaded, the missing information filled out, then uploaded using the File explorer in the Windows device portal.

{
  "MQTTUserName": "AdaFruitIO user",
  "MQTTPassword": "AIO Key",
  "MQTTClientID": "MQTTLoRaGateway",
  "MQTTServer": "io.adafruit.com",
  "Address": "LoRaIoT1",
  "Frequency": 915000000.0,
  "MessageHandlerAssembly": "Mqtt.IoTCore.FieldGateway.LoRa.Adafruit",
  "PlatformSpecificConfiguration": "mqttloragateway"
}

The application logs debugging information to the Windows 10 IoT Core ETW logging Microsoft-Windows-Diagnostics-LoggingChannel

MQTT LoRa Gateway with Adafruit.IO plug-in

The SaaS platform specific interface has gained an additional parameter for platform specific configuration.

namespace devMobile.Mqtt.IoTCore.FieldGateway
{
	using System;
	using Windows.Foundation.Diagnostics;

	using devMobile.IoT.Rfm9x;
	using MQTTnet;
	using MQTTnet.Client;

	public interface IMessageHandler
	{
		void Initialise(LoggingChannel logging, IMqttClient mqttClient, Rfm9XDevice rfm9XDevice,string platformSpecificConfiguration);

		void Rfm9XOnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e);

		void MqttApplicationMessageReceived(object sender, MqttApplicationMessageReceivedEventArgs e);

		void Rfm9xOnTransmit(object sender, Rfm9XDevice.OnDataTransmitedEventArgs e);
	}
}

This is used for the AdaFruit.IO GroupName so Adafruit.IO feed values are not all in a single group.

public class MessageHandler : IMessageHandler
	{
		private LoggingChannel Logging { get; set; }
		private IMqttClient MqttClient { get; set; }
		private Rfm9XDevice Rfm9XDevice { get; set; }
      private string PlatformSpecificConfiguration { get; set; }


      void IMessageHandler.Initialise(LoggingChannel logging, IMqttClient mqttClient, Rfm9XDevice rfm9XDevice, string platformSpecificConfiguration)
		{
			LoggingFields processInitialiseLoggingFields = new LoggingFields();

			this.Logging = logging;
			this.MqttClient = mqttClient;
			this.Rfm9XDevice = rfm9XDevice;
			this.PlatformSpecificConfiguration = platformSpecificConfiguration;
		}

		async void IMessageHandler.Rfm9XOnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e)
		{
			LoggingFields processReceiveLoggingFields = new LoggingFields();

			processReceiveLoggingFields.AddString("PacketSNR", e.PacketSnr.ToString("F1"));
			processReceiveLoggingFields.AddInt32("PacketRSSI", e.PacketRssi);
			processReceiveLoggingFields.AddInt32("RSSI", e.Rssi);

			string addressBcdText = BitConverter.ToString(e.Address);
			processReceiveLoggingFields.AddInt32("DeviceAddressLength", e.Address.Length);
			processReceiveLoggingFields.AddString("DeviceAddressBCD", addressBcdText);

			string payloadBcdText = BitConverter.ToString(e.Data);
			processReceiveLoggingFields.AddInt32("PayloadLength", e.Data.Length);
			processReceiveLoggingFields.AddString("DeviceAddressBCD", payloadBcdText);

			this.Logging.LogEvent("Rfm9XOnReceive", processReceiveLoggingFields, LoggingLevel.Information);
		}

		void IMessageHandler.MqttApplicationMessageReceived(object sender, MqttApplicationMessageReceivedEventArgs e)
		{
			LoggingFields processReceiveLoggingFields = new LoggingFields();

			processReceiveLoggingFields.AddString("ClientId", e.ClientId);
#if DEBUG
			processReceiveLoggingFields.AddString("Payload", e.ApplicationMessage.ConvertPayloadToString());
#endif
			processReceiveLoggingFields.AddString("QualityOfServiceLevel", e.ApplicationMessage.QualityOfServiceLevel.ToString());
			processReceiveLoggingFields.AddBoolean("Retain", e.ApplicationMessage.Retain);
			processReceiveLoggingFields.AddString("Topic", e.ApplicationMessage.Topic);

			this.Logging.LogEvent("MqttApplicationMessageReceived topic not processed", processReceiveLoggingFields, LoggingLevel.Error);
		}

		void IMessageHandler.Rfm9xOnTransmit(object sender, Rfm9XDevice.OnDataTransmitedEventArgs e)
		{
			this.Logging.LogMessage("Rfm9xOnTransmit", LoggingLevel.Information);
		}
	}
Adafruit.IO Group for a single field gateway
Group Setup

The message handler uploads all values in an inbound messages in one MQTT message using the AdaFruit.IO Group Feed format.

      async void IMessageHandler.Rfm9XOnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e)
      {
         LoggingFields processReceiveLoggingFields = new LoggingFields();
         char[] sensorReadingSeparators = { ',' };
         char[] sensorIdAndValueSeparators = { ' ' };

         processReceiveLoggingFields.AddString("PacketSNR", e.PacketSnr.ToString("F1"));
         processReceiveLoggingFields.AddInt32("PacketRSSI", e.PacketRssi);
         processReceiveLoggingFields.AddInt32("RSSI", e.Rssi);

         string addressBcdText = BitConverter.ToString(e.Address);
         processReceiveLoggingFields.AddInt32("DeviceAddressLength", e.Address.Length);
         processReceiveLoggingFields.AddString("DeviceAddressBCD", addressBcdText);

         string messageText;
         try
         {
            messageText = UTF8Encoding.UTF8.GetString(e.Data);
            processReceiveLoggingFields.AddString("MessageText", messageText);
         }
         catch (Exception ex)
         {
            processReceiveLoggingFields.AddString("Exception", ex.ToString());
            this.Logging.LogEvent("PayloadProcess failure converting payload to text", processReceiveLoggingFields, LoggingLevel.Warning);
            return;
         }

         // Chop up the CSV text
         string[] sensorReadings = messageText.Split(sensorReadingSeparators, StringSplitOptions.RemoveEmptyEntries);
         if (sensorReadings.Length < 1)
         {
            this.Logging.LogEvent("PayloadProcess payload contains no sensor readings", processReceiveLoggingFields, LoggingLevel.Warning);
            return;
         }

         JObject payloadJObject = new JObject();

         JObject feeds = new JObject();

         // Chop up each sensor read into an ID & value
         foreach (string sensorReading in sensorReadings)
         {
            string[] sensorIdAndValue = sensorReading.Split(sensorIdAndValueSeparators, StringSplitOptions.RemoveEmptyEntries);

            // Check that there is an id & value
            if (sensorIdAndValue.Length != 2)
            {
               this.Logging.LogEvent("PayloadProcess payload invalid format", processReceiveLoggingFields, LoggingLevel.Warning);
               return;
            }

            string sensorId = string.Concat(addressBcdText, sensorIdAndValue[0]);
            string value = sensorIdAndValue[1];

            feeds.Add(sensorId.ToLower(), value);
         }
         payloadJObject.Add("feeds", feeds);

         string topic = $"{MqttClient.Options.Credentials.Username}/groups/{PlatformSpecificConfiguration}";

         try
         {
            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topic)
               .WithPayload(JsonConvert.SerializeObject(payloadJObject))
               .WithAtLeastOnceQoS()
               .Build();
            Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync start", DateTime.UtcNow);
            await MqttClient.PublishAsync(message);
            Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync finish", DateTime.UtcNow);

            this.Logging.LogEvent("PublishAsync Adafruit payload", processReceiveLoggingFields, LoggingLevel.Information);
         }
         catch (Exception ex)
         {
            processReceiveLoggingFields.AddString("Exception", ex.ToString());
            this.Logging.LogEvent("PublishAsync Adafruit payload", processReceiveLoggingFields, LoggingLevel.Error);
         }
      }

The casing of User names (Must match exactly) and Group/Feed names (must be lower case) tripped me up yet again. The “automagic” provisioning of feeds does make setting up small scale systems easier, though I’m not certain how well it would scale.

Adafruit MQTT Cloud to Device Messaging

After getting MQ Telemetry Transport (MQTT) Device to Cloud (D2C) messaging working for AdaFruit.IO I have also got Cloud to Device (C2D) messaging working as well.

The MQTT broker, username, API key, client ID, optional group name (to keep MQTT aligned with REST API terminology), command topic and feed name are command line options.

The Adafruit IO MQTT documentation suggests an approach for naming topics which allows a bit more structure for feed (D2C and C2D) names than the REST API (which only does D2C).

class Program
{
	private static IMqttClient mqttClient = null;
	private static IMqttClientOptions mqttOptions = null;
	private static string server;
	private static string username;
	private static string password;
	private static string clientId;
	private static string commandTopic;
	private static string groupname;
	private static string feedname;

	static void Main(string[] args)
	{
		MqttFactory factory = new MqttFactory();
		mqttClient = factory.CreateMqttClient();

		if ((args.Length != 6) && (args.Length != 7))
		{
			Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID] [CommandTopic] [GroupName] [FeedName]");
			Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID] [CommandTopic] [FeedName]");
			Console.WriteLine("Press <enter> to exit");
			Console.ReadLine();
			return;
		}

		server = args[0];
		username = args[1];
		password = args[2];
		clientId = args[3];
		commandTopic = args[4];
		if (args.Length == 6)
		{
			feedname = args[5].ToLower();
			Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} CommandTopic:{commandTopic} Feedname:{feedname}");
		}

		if (args.Length == 7)
		{
			groupname = args[5].ToLower();
			feedname = args[6].ToLower();
			Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} CommandTopic:{commandTopic} Groupname:{groupname} Feedname:{feedname}");
		}

		mqttOptions = new MqttClientOptionsBuilder()
			.WithTcpServer(server)
			.WithCredentials(username, password)
			.WithClientId(clientId)
			.WithTls()
			.Build();

		mqttClient.Disconnected += MqttClient_Disconnected;
		mqttClient.ConnectAsync(mqttOptions).Wait();
		mqttClient.ApplicationMessageReceived += MqttClient_ApplicationMessageReceived;

		// Adafruit.IO format for topics which are called feeds
		string topic = string.Empty;

		if (args.Length == 6)
		{
			topic = $"{username}/feeds/{feedname}";				
		}

		if (args.Length == 7)
		{
			topic = $"{username}/feeds/{groupname}.{feedname}";
		}

		mqttClient.SubscribeAsync(commandTopic, MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce).GetAwaiter().GetResult();

		while (true)
		{
			string value = "22." + DateTime.UtcNow.Millisecond.ToString();
			Console.WriteLine($"Topic:{topic} Value:{value}");

			var message = new MqttApplicationMessageBuilder()
				.WithTopic(topic)
				.WithPayload(value)
				.WithQualityOfServiceLevel(MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce)
			.WithRetainFlag()
			.Build();

			Console.WriteLine("PublishAsync start");
			mqttClient.PublishAsync(message).Wait();
			Console.WriteLine("PublishAsync finish");

			Thread.Sleep(30100);
		}
	}

	private static void MqttClient_ApplicationMessageReceived(object sender, MqttApplicationMessageReceivedEventArgs e)
	{
		Console.WriteLine($"ClientId:{e.ClientId} Topic:{e.ApplicationMessage.Topic} Payload:{e.ApplicationMessage.ConvertPayloadToString()}");
	}

	private static async void MqttClient_Disconnected(object sender, MqttClientDisconnectedEventArgs e)
	{
		Debug.WriteLine("Disconnected");
		await Task.Delay(TimeSpan.FromSeconds(5));

		try
		{
			await mqttClient.ConnectAsync(mqttOptions);
		}
		catch (Exception ex)
		{
			Debug.WriteLine("Reconnect failed {0}", ex.Message);
		}
	}
}

I configured a slider on the dashboard for my home called “setpoint” (yet again I was tripped up “automatically” camel casing the name because I’m a C# developer) which my MQTT client subscribed to.

AdaFruit.IO Home monitoring dashboard
setpoint feed configuration

After figuring out the format of the command topic I found that when the slider was moved the MQTT client subscription event fired reliably.

AdaFruit Client showing the setpoint value change notifications

Overall the process went pretty well, though the manual configuration of the subscriptions to AdaFruit.IO feeds could become a bit of a problem at scale.

Ask Sensors with MQTTnet

After a 6 month pause I’m back working on my Message Queue Telemetry Transport(MQTT) LoRa gateway.

As I’m testing my Message Queue Telemetry Transport(MQTT) LoRa gateway I’m building a proof of concept(PoC) .Net core console application for each IoT platform I would like to support.

This PoC was to confirm that I could connect to the Ask Sensors MQTT API then format topics and payloads correctly.

Console test application

The MQTT broker, username, password, and client ID are command line options.

class Program
{
	private static IMqttClient mqttClient = null;
	private static IMqttClientOptions mqttOptions = null;
	private static string server;
	private static string username;
	private static string apiKey;
	private static string clientID;

	static void Main(string[] args)
	{
		MqttFactory factory = new MqttFactory();
		mqttClient = factory.CreateMqttClient();
		bool heatPumpOn = false;

		if (args.Length != 4)
		{
			Console.WriteLine("[MQTT Server] [UserName] [APIKey] [ClientID]");
			Console.WriteLine("Press <enter> to exit");
			Console.ReadLine();
			return;
		}

		server = args[0];
		username = args[1];
		apiKey = args[2];
		clientID = args[3];

		Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientID}");

		mqttOptions = new MqttClientOptionsBuilder()
			.WithTcpServer(server)
			.WithCredentials(username, "")
			.WithClientId(clientID)
			//.WithTls() // This is a bit of a worry
			.Build();

		mqttClient.ApplicationMessageReceived += MqttClient_ApplicationMessageReceived;
		mqttClient.Disconnected += MqttClient_Disconnected;
		mqttClient.ConnectAsync(mqttOptions).Wait();

		// AskSensors formatted client state update topic
		string stateTopic = $"{username}/{apiKey}";

		while (true)
		{
			string payloadText;
			double temperature = 22.0 + (DateTime.UtcNow.Millisecond / 1000.0);
			double humidity = 50 + (DateTime.UtcNow.Millisecond / 100.0);
			double speed = 10 + (DateTime.UtcNow.Millisecond / 100.0);
			Console.WriteLine($"Topic:{stateTopic} Temperature:{temperature:0.00} Humidity:{humidity:0} HeatPumpOn:{heatPumpOn}");

			// First JSON attempt didn't work
			payloadText = @"{""Humidity"":55}";

            // Second attempt worked
            payloadText = $"module1=22";

            // Third attempt with "real" values injected
            payloadText = $"module1={temperature}&m2={humidity}";

            var message = new MqttApplicationMessageBuilder()
					.WithTopic(stateTopic)
					.WithPayload(payloadText)
					.WithQualityOfServiceLevel(global::MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce)
				   .WithExactlyOnceQoS()
				   //.WithAtLeastOnceQoS()
				   //.WithRetainFlag()
				   .Build();

				Console.WriteLine("PublishAsync start");
				mqttClient.PublishAsync(message).Wait();
				Console.WriteLine("PublishAsync finish");

				Thread.Sleep(30100);
			}
		}

	private static async void MqttClient_Disconnected(object sender, MqttClientDisconnectedEventArgs e)
	{
		Debug.WriteLine("Disconnected");
		await Task.Delay(TimeSpan.FromSeconds(5));

		try
		{
			await mqttClient.ConnectAsync(mqttOptions);
		}
		catch (Exception ex)
		{
			Debug.WriteLine("Reconnect failed {0}", ex.Message);
		}
	}

The Ask Sensors screen designer has 8 different types of Graph (line, bar, scatter, gauge, table, binary. digital, map)

Ask sensors dashboard configuration was fairly simple, though sequential numbering of inputs (modules) might require some mapping code.

Overall the initial configuration went smoothly after I figured out the payload format (not JSON), though the functionality (of a free subscription) did appear to be quite limited.

Since I first started building my MQTT gateway there have been several breaking updates to the MQTTNet API which so I will have to refresh all the applications in my solution.

Maduino LoRa Air Temperature and Soil Moisture

This is a demo MakerFabs Maduino LoRa Radio 868MHz client (based on Maduino LoRa 868MHz example) that uploads telemetry data to my Windows 10 IoT Core on Raspberry PI AdaFruit.IO and Azure IoT Hub field gateways.

The code is available on github

Sample hardware
Azure IoT Central data visualisation

The Maduino device in the picture is a custom version with an onboard Microchip ATSHA204 crypto and authentication chip (currently only use for the unique 72 bit serial number) and a voltage divider connected to the analog pin A6 to monitor the battery voltage.

There are compile time options ATSHA204 & BATTERY_VOLTAGE_MONITOR which can be used to selectively enable this functionality.

I use the Arduino lowpower library to aggressively sleep the device between measurements

// Adjust the delay so period is close to desired sec as possible, first do 8sec chunks. 
  int delayCounter = SensorUploadDelay / 8 ;
  for( int i = 0 ; i < delayCounter ; i++ )
  {
     LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);  
  }
  
  // Then to 4 sec chunk
  delayCounter =  ( SensorUploadDelay % 8 ) / 4;
  for( int i = 0 ; i < delayCounter ; i++ )
  {
     LowPower.powerDown(SLEEP_4S, ADC_OFF, BOD_OFF);  
  }

  // Then to 2 sec chunk
  delayCounter =  ( SensorUploadDelay % 4 ) / 2 ;
  for( int i = 0 ; i < delayCounter ; i++ )
  {
     LowPower.powerDown(SLEEP_2S, ADC_OFF, BOD_OFF);  
  }

  // Then to 1 sec chunk
  delayCounter =  ( SensorUploadDelay % 2 ) ;
  for( int i = 0 ; i < delayCounter ; i++ )
  {
     LowPower.powerDown(SLEEP_1S, ADC_OFF, BOD_OFF);  
  }
}

I use a spare digital PIN for powering the soil moisture probe so it can be powered down when not in use. I have included a short delay after powering up the device to allow the reading to settle.

  // Turn on soil mosture sensor, take reading then turn off to save power
  digitalWrite(SoilMoistureSensorEnablePin, HIGH);
  delay(SoilMoistureSensorEnableDelay);
  int soilMoistureADCValue = analogRead(SoilMoistureSensorPin);
  digitalWrite(SoilMoistureSensorEnablePin, LOW);
  int soilMoisture = map(soilMoistureADCValue,SoilMoistureSensorMinimum,SoilMoistureSensorMaximum, SoilMoistureValueMinimum, SoilMoistureValueMaximum); 
  PayloadAdd( "s", soilMoisture, false);

Bill of materials (Prices Nov 2019)

  • Maduino LoRa Radion (868MHz) 18.90
  • SHT20 I2C Temperature & Humidity Sensor (Waterproof Probe) USD22.50
  • Pinotech SoilWatch 10 – Soil moisture sensor USD23
  • Elecrow 1 Watt solar panel with wires USD3.80
  • 500 mAh LI-Ion battery

The software could easily be modified to support additional sensors.