Bosch IoT Suite with MQTTnet

As I’m testing my Message Queue Telemetry Transport(MQTT) LoRa gateway I’m building a proof of concept(PoC) .Net core console application for each IoT platform I would like to support.

This PoC was to confirm that I could connect to the Bosch IoT Suite MQTT API then format topics and payloads correctly.

MQTTNet Console Client

The Bosch IoT Hub MQTT broker, username, password, and clientID are the required command line parameters. For this PoC I ran out of time to get cloud to device (C2D) messaging or any presentation functionality working.

/*
    Copyright ® 2019 December devMobile Software, All Rights Reserved
 
    MIT License

    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"), to deal
    in the Software without restriction, including without limitation the rights
    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    copies of the Software, and to permit persons to whom the Software is
    furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in all
    copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
    SOFTWARE

	 A quick and dirty test client to explore how BoschIoT Suite MQTT connectivity works
 */
namespace devMobile.Mqtt.TestClient.BoschIoTSuite
{
   using System;
   using System.Diagnostics;
   using System.Threading;
   using System.Threading.Tasks;

   using MQTTnet;
   using MQTTnet.Client;
   using MQTTnet.Client.Disconnecting;
   using MQTTnet.Client.Options;
   using MQTTnet.Client.Receiving;
   using Newtonsoft.Json;
   using Newtonsoft.Json.Linq;

   class Program
   {
      private static IMqttClient mqttClient = null;
      private static IMqttClientOptions mqttOptions = null;
      private static string server;
      private static string username;
      private static string password;
      private static string clientId;

      static void Main(string[] args)
      {
         MqttFactory factory = new MqttFactory();
         mqttClient = factory.CreateMqttClient();

         if (args.Length != 4) 
         {
            Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID]");
            Console.WriteLine("Press <enter> to exit");
            Console.ReadLine();
            return;
         }

         server = args[0];
         username = args[1];
         password = args[2];
         clientId = args[3];

         mqttOptions = new MqttClientOptionsBuilder()
            .WithTcpServer(server)
            .WithCredentials(username, password)
            .WithClientId(clientId)
            .WithTls()
            .Build();

         mqttClient.UseDisconnectedHandler(new MqttClientDisconnectedHandlerDelegate(e => MqttClient_Disconnected(e)));
         mqttClient.UseApplicationMessageReceivedHandler(new MqttApplicationMessageReceivedHandlerDelegate(e => MqttClient_ApplicationMessageReceived(e)));
         mqttClient.ConnectAsync(mqttOptions).Wait();

         string topicD2C = "telemetry";

         while (true)
         {
            JObject payloadJObject = new JObject();

            payloadJObject.Add("OfficeTemperature", "22." + DateTime.UtcNow.Millisecond.ToString());
            payloadJObject.Add("OfficeHumidity", (DateTime.UtcNow.Second + 40).ToString());

            string payload = JsonConvert.SerializeObject(payloadJObject);
            Console.WriteLine($"Topic:{topicD2C} Payload:{payload}");

            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topicD2C)
               .WithPayload(payload)
               .WithAtMostOnceQoS() // Anthing but this causes timeout
               .WithRetainFlag()
            .Build();

            Console.WriteLine("PublishAsync start");
            mqttClient.PublishAsync(message).Wait();
            Console.WriteLine("PublishAsync finish");

            Thread.Sleep(30100);
         }
      }

      private static void MqttClient_ApplicationMessageReceived(MqttApplicationMessageReceivedEventArgs e)
      {
         Console.WriteLine($"ClientId:{e.ClientId} Topic:{e.ApplicationMessage.Topic} Payload:{e.ApplicationMessage.ConvertPayloadToString()}");
      }

      private static async void MqttClient_Disconnected(MqttClientDisconnectedEventArgs e)
      {
         Debug.WriteLine("Disconnected");
         await Task.Delay(TimeSpan.FromSeconds(5));

         try
         {
            await mqttClient.ConnectAsync(mqttOptions);
         }
         catch (Exception ex)
         {
            Debug.WriteLine("Reconnect failed {0}", ex.Message);
         }
      }
   }
}

The bosch IoT Hub device configuration was via a swagger API but I need to spend some more time figuring out how to configure the data analysis and presentation tools.

I adapted the steps in the IoT Hub Documentation for Sending Device Data using MQTT. The first step was to create a free Hub subscription.

IoT Hub Subscription

Then using the device registry swagger UI page to add a new device.

Device Registry Swagger UI

After a couple of failed attempts I worked out the format of the Authorisation details (I think the username format in the online documentation might be wrong)

Swagger UI Authorisation form
Querying the available devices

Of the 10+ SaaS IoT services I have setup the Bosch IoT Suite was the hardest to get working. I think this was becuase it is meant to be managed via the API from a in-house application. In a future post I’ll get configure the cloud to device messaging, plus analysis and display functionality.

Armtronix IA005 SX1276 loRa node

A month ago I ordered a pair of IA005: SX1276 Lora node STM32F103 devices from the Armtronix shop on Tindie for evaluation. At USD18 each these devices were competitively priced and I was interested in trialling another maple like device.

Bill of materials (Prices as at December 2019)

  • IA005 SX1276 loRa node USD36 (USD18 each)
  • Grove – Temperature&Humidity Sensor USD11.5
  • Grove – 4 pin Female Jumper to Grove 4 pin Conversion Cable USD3.90
Armtronix device with Seeedstudio temperature & humidity sensor

I used a modified version of my Arduino client code which worked after I got the pins sorted and the female jumper sockets in the right order.

/*
  Copyright ® 2019 December devMobile Software, All Rights Reserved

  THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY
  KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
  IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR
  PURPOSE.

  You can do what you want with this code, acknowledgment would be nice.

  http://www.devmobile.co.nz

*/
#include <stdlib.h>
#include <LoRa.h>
#include <TH02_dev.h>

//#define DEBUG
//#define DEBUG_TELEMETRY
//#define DEBUG_LORA

// LoRa field gateway configuration (these settings must match your field gateway)
const char FieldGatewayAddress[] = {"LoRaIoT1"};
const char DeviceAddress[] = {"ArmTronix01"};
const float FieldGatewayFrequency =  915000000.0;
const byte FieldGatewaySyncWord = 0x12 ;

// Payload configuration
const int ChipSelectPin = PA4;
const int InterruptPin = PA11;
const int ResetPin = PC13;

// LoRa radio payload configuration
const byte SensorIdValueSeperator = ' ' ;
const byte SensorReadingSeperator = ',' ;
const int LoopSleepDelaySeconds = 30 ;

const byte PayloadSizeMaximum = 64 ;
byte payload[PayloadSizeMaximum];
byte payloadLength = 0 ;


void setup()
{
  Serial.begin(9600);
#ifdef DEBUG
  while (!Serial);
#endif
  Serial.println("Setup called");

  Serial.println("LoRa setup start");
  
  // override the default chip select and reset pins
  LoRa.setPins(ChipSelectPin, ResetPin, InterruptPin); 
  if (!LoRa.begin(FieldGatewayFrequency))
  {
    Serial.println("LoRa begin failed");
    while (true); // Drop into endless loop requiring restart
  }

  // Need to do this so field gateways pays attention to messsages from this device
  LoRa.enableCrc();
  LoRa.setSyncWord(FieldGatewaySyncWord);

#ifdef DEBUG_LORA
  LoRa.dumpRegisters(Serial);
#endif
  Serial.println("LoRa Setup done.");

  // Configure the Seeedstudio TH02 temperature & humidity sensor
  Serial.println("TH02 setup start");
  TH02.begin();
  delay(100);
  Serial.println("TH02 setup done");

  PayloadHeader((byte*)FieldGatewayAddress,strlen(FieldGatewayAddress), (byte*)DeviceAddress, strlen(DeviceAddress));

  Serial.println("Setup done");
  Serial.println();
}


void loop()
{
  float temperature ;
  float humidity ;

  Serial.println("Loop called");

  PayloadReset();

  // Read the temperature & humidity & battery voltage values then display nicely
  temperature = TH02.ReadTemperature();
  Serial.print("T:");
  Serial.print( temperature, 1 ) ;
  Serial.println( "C " ) ;

  PayloadAdd( "T", temperature, 1);

  humidity = TH02.ReadHumidity();
  Serial.print("H:");
  Serial.print( humidity, 0 ) ;
  Serial.println( "% " ) ;

  PayloadAdd( "H", humidity, 0) ;

#ifdef DEBUG_TELEMETRY
  Serial.print( "RFM9X/SX127X Payload len:");
  Serial.print( payloadLength );
  Serial.println( " bytes" );
#endif

  LoRa.beginPacket();
  LoRa.write( payload, payloadLength );
  LoRa.endPacket();

  Serial.println("Loop done");
  Serial.println();
  delay(LoopSleepDelaySeconds * 1000l);
}


void PayloadHeader( byte *to, byte toAddressLength, byte *from, byte fromAddressLength)
{
  byte addressesLength = toAddressLength + fromAddressLength ;

#ifdef DEBUG_TELEMETRY
  Serial.print("PayloadHeader- ");
  Serial.print( "To len:");
  Serial.print( toAddressLength );
  Serial.print( " From len:");
  Serial.print( fromAddressLength );
  Serial.print( " Header len:");
  Serial.print( addressesLength );
  Serial.println( );
#endif

  payloadLength = 0 ;

  // prepare the payload header with "To" Address length (top nibble) and "From" address length (bottom nibble)
  payload[payloadLength] = (toAddressLength << 4) | fromAddressLength ;
  payloadLength += 1;

  // Copy the "To" address into payload
  memcpy(&payload[payloadLength], to, toAddressLength);
  payloadLength += toAddressLength ;

  // Copy the "From" into payload
  memcpy(&payload[payloadLength], from, fromAddressLength);
  payloadLength += fromAddressLength ;
}


void PayloadAdd( const char *sensorId, float value, byte decimalPlaces)
{
  byte sensorIdLength = strlen( sensorId ) ;

#ifdef DEBUG_TELEMETRY
  Serial.print("PayloadAdd-float ");
  Serial.print( "SensorId:");
  Serial.print( sensorId );
  Serial.print( " Len:");
  Serial.print( sensorIdLength );
  Serial.print( " Value:");
  Serial.print( value, decimalPlaces );
  Serial.print( " payloadLen:");
  Serial.print( payloadLength);
#endif

  memcpy( &payload[payloadLength], sensorId,  sensorIdLength) ;
  payloadLength += sensorIdLength ;
  payload[ payloadLength] = SensorIdValueSeperator;
  payloadLength += 1 ;
  payloadLength += strlen( dtostrf(value, -1, decimalPlaces, (char *)&payload[payloadLength]));
  payload[ payloadLength] = SensorReadingSeperator;
  payloadLength += 1 ;
  
#ifdef DEBUG_TELEMETRY
  Serial.print( " payloadLen:");
  Serial.println( payloadLength);
#endif
}


void PayloadAdd( const char *sensorId, int value )
{
  byte sensorIdLength = strlen( sensorId ) ;

#ifdef DEBUG_TELEMETRY
  Serial.print("PayloadAdd-int ");
  Serial.print( "SensorId:");
  Serial.print( sensorId );
  Serial.print( " Len:");
  Serial.print( sensorIdLength );
  Serial.print( " Value:");
  Serial.print( value );
  Serial.print( " payloadLen:");
  Serial.print( payloadLength);
#endif  

  memcpy( &payload[payloadLength], sensorId,  sensorIdLength) ;
  payloadLength += sensorIdLength ;
  payload[ payloadLength] = SensorIdValueSeperator;
  payloadLength += 1 ;
  payloadLength += strlen( itoa( value,(char *)&payload[payloadLength],10));
  payload[ payloadLength] = SensorReadingSeperator;
  payloadLength += 1 ;
  
#ifdef DEBUG_TELEMETRY
  Serial.print( " payloadLen:");
  Serial.println( payloadLength);
#endif
}


void PayloadAdd( const char *sensorId, unsigned int value )
{
  byte sensorIdLength = strlen( sensorId ) ;

#ifdef DEBUG_TELEMETRY
  Serial.print("PayloadAdd-unsigned int ");
  Serial.print( "SensorId:");
  Serial.print( sensorId );
  Serial.print( " Len:");
  Serial.print( sensorIdLength );
  Serial.print( " Value:");
  Serial.print( value );
  Serial.print( " payloadLen:");
  Serial.print( payloadLength);
#endif  

  memcpy( &payload[payloadLength], sensorId,  sensorIdLength) ;
  payloadLength += sensorIdLength ;
  payload[ payloadLength] = SensorIdValueSeperator;
  payloadLength += 1 ;
  payloadLength += strlen( utoa( value,(char *)&payload[payloadLength],10));
  payload[ payloadLength] = SensorReadingSeperator;
  payloadLength += 1 ;

#ifdef DEBUG_TELEMETRY
  Serial.print( " payloadLen:");
  Serial.println( payloadLength);
#endif
}


void PayloadReset()
{
  byte fromAddressLength = payload[0] & 0xf ;
  byte toAddressLength = payload[0] >> 4 ;
  byte addressesLength = toAddressLength + fromAddressLength ;

  payloadLength = addressesLength + 1;

#ifdef DEBUG_TELEMETRY
  Serial.print("PayloadReset- ");
  Serial.print( "To len:");
  Serial.print( toAddressLength );
  Serial.print( " From len:");
  Serial.print( fromAddressLength );
  Serial.print( " Header len:");
  Serial.println( addressesLength );
#endif
}

To get the application to download I had to configure the board in the Arduino IDE

Then change the jumpers

Initially I had some problems deploying my software because I hadn’t followed the instructions (the wiki everyone referred to appeared to be offline) and run the installation batch file (New dev machine since my previous maple based project).

15:40:56.207 -> LoRa Setup done.
15:40:56.207 -> TH02 setup start
15:40:56.307 -> TH02 setup done
15:40:56.307 -> PayloadHeader- To len:8 From len:11 Header len:19
15:40:56.354 -> Setup done
15:40:56.354 -> 
15:40:56.354 -> Loop called
15:40:56.354 -> PayloadReset- To len:8 From len:11 Header len:19
15:40:56.408 -> T:23.9C 
15:40:56.408 -> PayloadAdd-float SensorId:T Len:1 Value:23.9 payloadLen:20 payloadLen:27
15:40:56.508 -> H:70% 
15:40:56.508 -> PayloadAdd-float SensorId:H Len:1 Value:70 payloadLen:27 payloadLen:32
15:40:56.608 -> RFM9X/SX127X Payload len:32 bytes
15:40:56.655 -> Loop done
15:40:56.655 -> 
15:41:26.647 -> Loop called
15:41:26.647 -> PayloadReset- To len:8 From len:11 Header len:19
15:41:26.684 -> T:24.0C 
15:41:26.730 -> PayloadAdd-float SensorId:T Len:1 Value:24.0 payloadLen:20 payloadLen:27
15:41:26.784 -> H:69% 
15:41:26.784 -> PayloadAdd-float SensorId:H Len:1 Value:69 payloadLen:27 payloadLen:32
15:41:26.884 -> RFM9X/SX127X Payload len:32 bytes
15:41:26.931 -> Loop done
15:41:26.931 -> 
15:41:56.904 -> Loop called
15:41:56.904 -> PayloadReset- To len:8 From len:11 Header len:19
15:41:56.948 -> T:24.1C 
15:41:56.982 -> PayloadAdd-float SensorId:T Len:1 Value:24.1 payloadLen:20 payloadLen:27
15:41:57.054 -> H:69% 
15:41:57.054 -> PayloadAdd-float SensorId:H Len:1 Value:69 payloadLen:27 payloadLen:32
15:41:57.157 -> RFM9X/SX127X Payload len:32 bytes
15:41:57.191 -> Loop done
15:41:57.191 -> 
15:42:27.211 -> Loop called
15:42:27.211 -> PayloadReset- To len:8 From len:11 Header len:19
15:42:27.258 -> T:24.1C 
15:42:27.258 -> PayloadAdd-float SensorId:T Len:1 Value:24.1 payloadLen:20 payloadLen:27
15:42:27.343 -> H:69% 
15:42:27.343 -> PayloadAdd-float SensorId:H Len:1 Value:69 payloadLen:27 payloadLen:32
15:42:27.427 -> RFM9X/SX127X Payload len:32 bytes
15:42:27.481 -> Loop done
15:42:27.481 -> 
15:42:57.504 -> Loop called
15:42:57.504 -> PayloadReset- To len:8 From len:11 Header len:19
15:42:57.504 -> T:24.1C 
15:42:57.550 -> PayloadAdd-float SensorId:T Len:1 Value:24.1 payloadLen:20 payloadLen:27
15:42:57.604 -> H:68% 
15:42:57.604 -> PayloadAdd-float SensorId:H Len:1 Value:68 payloadLen:27 payloadLen:32
15:42:57.704 -> RFM9X/SX127X Payload len:32 bytes
15:42:57.755 -> Loop done
15:42:57.755 -> 

I configured the device to upload to my Azure IoT Hub/Azure IoT Central gateway and it has been running reliably for a couple of days.

Azure IoT Central temperature and humidity values

Initially I had some configuration problems but I contacted Armtronix support and they promptly provided a couple of updated links for product and device documentation.

wolkabout with MQTTnet

As I’m testing my Message Queue Telemetry Transport(MQTT) LoRa gateway I’m building a proof of concept(PoC) .Net core console application for each IoT platform I would like to support.

This PoC was to confirm that I could connect to the walkabout MQTT API then format topics and payloads correctly.

MQTTNet Console Client

The walkabout MQTT broker, username, API Key, and device ID are the required command line parameters. For this PoC I couldn’t get cloud to device (C2D) or Transport Layer Security(TLS) working so will have to do some more research.

namespace devmobile.Mqtt.TestClient.WolkAbout
{
   using System;
   using System.Diagnostics;
   using System.Threading;
   using System.Threading.Tasks;

   using MQTTnet;
   using MQTTnet.Client;
   using MQTTnet.Client.Disconnecting;
   using MQTTnet.Client.Options;

   using Newtonsoft.Json;
   using Newtonsoft.Json.Linq;

   class Program
   {
      private static IMqttClient mqttClient = null;
      private static IMqttClientOptions mqttOptions = null;
      private static string server;
      private static string username;
      private static string apiKey;
      private static string clientID;

      static void Main(string[] args)
      {
         MqttFactory factory = new MqttFactory();
         mqttClient = factory.CreateMqttClient();

         if ((args.Length != 4) )
            {
            Console.WriteLine("[MQTT Server] [UserName] [APIKey] [ClientID]");
            Console.WriteLine("Press <enter> to exit");
            Console.ReadLine();
            return;
         }

         server = args[0];
         username = args[1];
         apiKey = args[2];
         clientID = args[3];

         Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientID}");

         // wolkabout formatted client state update topic
         string topicD2C = $"readings/{username}/";

         mqttOptions = new MqttClientOptionsBuilder()
            .WithTcpServer(server)
            .WithCredentials(username, apiKey)
            .WithClientId(clientID)
            //.WithTls()
            .Build();

         mqttClient.UseDisconnectedHandler(new MqttClientDisconnectedHandlerDelegate(e => MqttClient_Disconnected(e)));
         mqttClient.ConnectAsync(mqttOptions).Wait();

         while (true)
         {
            JObject payloadJObject = new JObject();

            double temperature = 22.0 + (DateTime.UtcNow.Millisecond / 1000.0);
            double humidity = 50 + (DateTime.UtcNow.Millisecond / 100.0);

            payloadJObject.Add("Temperature", temperature);
            payloadJObject.Add("Humidity", humidity);

            string payload = JsonConvert.SerializeObject(payloadJObject);
            Console.WriteLine($"Topic:{topicD2C} Payload:{payload}");

            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topicD2C)
               .WithPayload(payload)
               .WithAtLeastOnceQoS()
               .Build();

            Console.WriteLine("PublishAsync start");
            mqttClient.PublishAsync(message).Wait();
            Console.WriteLine("PublishAsync finish");

            Thread.Sleep(30100);
         }
      }

      private static async void MqttClient_Disconnected(MqttClientDisconnectedEventArgs e)
      {
         Debug.WriteLine("Disconnected");
         await Task.Delay(TimeSpan.FromSeconds(5));

         try
         {
            await mqttClient.ConnectAsync(mqttOptions);
         }
         catch (Exception ex)
         {
            Debug.WriteLine("Reconnect failed {0}", ex.Message);
         }
      }
   }

The walkabout device configuration was relatively easy but I need watch the instructional videos again to better understand the device and data semantics relationship.

Data semantics configuration
Devices setup
Device Setup
My first dashboard

SmartWorks with MQTTnet

As I’m testing my Message Queue Telemetry Transport(MQTT) LoRa gateway I’m building a proof of concept(PoC) .Net core console application for each IoT platform I would like to support.

This PoC was to confirm that I could connect to the SmartWorks (formerly Carriots) MQTT API then format topics and payloads correctly.

MQTTNet Console Client

The SmartWorks MQTT broker, username, and device ID are the required command line parameters. I didn’t notice any configuration options for cloud to device (C2D) messaging which maybe due to my device configuration or the free trial I was using.

namespace devMobile.Mqtt.TestClient.SmartWorks
{
   using System;
   using System.Diagnostics;
   using System.Threading;
   using System.Threading.Tasks;

   using MQTTnet;
   using MQTTnet.Client;
   using MQTTnet.Client.Disconnecting;
   using MQTTnet.Client.Options;
   using MQTTnet.Client.Receiving;
   using Newtonsoft.Json;
   using Newtonsoft.Json.Linq;

   class Program
   {
      private static IMqttClient mqttClient = null;
      private static IMqttClientOptions mqttOptions = null;
      private static string server;
      private static string username;
      private static string clientId;
      private static string commandTopic;
      private static string groupname;
      private static string feedname;

      static void Main(string[] args)
      {
         MqttFactory factory = new MqttFactory();
         mqttClient = factory.CreateMqttClient();

         if (args.Length != 3) 
         {
            Console.WriteLine("[MQTT Server] [UserName] [ClientID]");
            Console.WriteLine("Press <enter> to exit");
            Console.ReadLine();
            return;
         }

         server = args[0];
         username = args[1];
         clientId = args[2];

         mqttOptions = new MqttClientOptionsBuilder()
            .WithTcpServer(server)
            .WithCredentials(username, "")
            .WithClientId(clientId)
            .WithTls()
            .Build();

         mqttClient.UseDisconnectedHandler(new MqttClientDisconnectedHandlerDelegate(e => MqttClient_Disconnected(e)));
         mqttClient.ConnectAsync(mqttOptions).Wait();

         // Adafruit.IO format for topics which are called feeds
         string topicD2C = $"{username}/streams";

         while (true)
         {
            JObject payloadJObject = new JObject();

            payloadJObject.Add("at", "now");
            payloadJObject.Add("device", clientId);
            payloadJObject.Add("protocol", "v2");

            double temperature = 22.0 + (DateTime.UtcNow.Millisecond / 1000.0);
            double humidity = 50 + (DateTime.UtcNow.Millisecond / 100.0);

            JObject dataJObject = new JObject();
            dataJObject.Add("OfficeTemperature", temperature);
            dataJObject.Add("OfficeHumidity", humidity);

            payloadJObject.Add("data", dataJObject);

            string payload = JsonConvert.SerializeObject(payloadJObject);
            Console.WriteLine($"Topic:{topicD2C} Payload:{payload}");

            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topicD2C)
               .WithPayload(payload)
               .WithAtLeastOnceQoS()
            .Build();

            Console.WriteLine("PublishAsync start");
            mqttClient.PublishAsync(message).Wait();
            Console.WriteLine("PublishAsync finish");

            Thread.Sleep(30100);
         }
      }

      private static async void MqttClient_Disconnected(MqttClientDisconnectedEventArgs e)
      {
         Debug.WriteLine("Disconnected");
         await Task.Delay(TimeSpan.FromSeconds(5));

         try
         {
            await mqttClient.ConnectAsync(mqttOptions);
         }
         catch (Exception ex)
         {
            Debug.WriteLine("Reconnect failed {0}", ex.Message);
         }
      }
   }
}

The ThingsBoard device configuration was relatively easy with convenient buttons to copy the Device ID (Client ID in test client) and Access Token (UserName in test client). I need to revisit the Device and Group configuration to see if I can make the automatically generated names more user friendly.

Devices configuration

The Device configuration form has a tab which has a link for the “Data Streams” form which was useful for debugging.

Device configuration

I have emailed SmartWorks support about a free trial of their dashboard product as it is not available in the free trial.

Device data stream query form

Overall the initial configuration went smoothly but the lack of any dashboard functionality in the free trial was quite limiting.

Azure IoT Hub SAS Tokens revisited yet again

Based my previous post on SAS Token Expiry I wrote a test harness to better understand DateTimeOffset

using System;

namespace UnixEpochTester
{
   class Program
   {
      static void Main(string[] args)
      {
         Console.WriteLine($"DIY                {new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)}");
         Console.WriteLine($"DateTime.UnixEpoch {DateTime.UnixEpoch} {DateTime.UnixEpoch.Kind}");
         Console.WriteLine();

         TimeSpan fromUnixEpochNow = DateTime.UtcNow - DateTime.UnixEpoch;
         Console.WriteLine($"Epoc now {fromUnixEpochNow} {fromUnixEpochNow.TotalSeconds.ToString("f0")} sec");
         Console.WriteLine();

         TimeSpan fromUnixEpochFixed = new DateTime(2019, 11, 30, 2, 0, 0, DateTimeKind.Utc) - DateTime.UnixEpoch;
         Console.WriteLine($"Epoc  {fromUnixEpochFixed} {fromUnixEpochFixed.TotalSeconds.ToString("f0")} sec");
         Console.WriteLine();

         DateTimeOffset dateTimeOffset = new DateTimeOffset( new DateTime( 2019,11,30,2,0,0, DateTimeKind.Utc));
         Console.WriteLine($"Epoc DateTimeOffset {fromUnixEpochFixed} {dateTimeOffset.ToUnixTimeSeconds()}");
         Console.WriteLine();

         TimeSpan fromEpochStart = new DateTime(2019, 11, 30, 2, 0, 0, DateTimeKind.Utc) - DateTime.UnixEpoch;
         Console.WriteLine($"Epoc DateTimeOffset {fromEpochStart} {fromEpochStart.TotalSeconds.ToString("F0")}");
         Console.WriteLine();


         // https://www.epochconverter.com/ matches
         // https://www.unixtimestamp.com/index.php matches

         Console.WriteLine("Press ENTER to exit");
         Console.ReadLine();
      }
   }
}

I validated my numbers against a couple of online calculators and they matched which was a good start.

DateTimeOffset test harness

As I was testing my Azure MQTT Test Client I had noticed some oddness with MQTT connection timeouts.

string token = generateSasToken($"{server}/devices/{clientId}", password, "", new TimeSpan(0,5,0));
1/12/2019 1:29:52 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.391","OfficeHumidity":"93"}]
1/12/2019 1:30:22 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.29","OfficeHumidity":"64"}]
...
1/12/2019 1:43:56 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.591","OfficeHumidity":"98"}]
1/12/2019 1:44:26 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.754","OfficeHumidity":"68"}]


string token = generateSasToken($"{server}/devices/{clientId}", password, "", new TimeSpan(0,5,0));
1/12/2019 1:29:52 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.391","OfficeHumidity":"93"}]
1/12/2019 1:30:22 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.29","OfficeHumidity":"64"}]
...
1/12/2019 2:01:37 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.334","OfficeHumidity":"79"}]
1/12/2019 2:02:07 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.503","OfficeHumidity":"49"}]


string token = generateSasToken($"{server}/devices/{clientId}", password, "", new TimeSpan(0,5,0));
2/12/2019 9:27:21 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.196","OfficeHumidity":"61"}]
2/12/2019 9:27:51 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.788","OfficeHumidity":"91"}]
...
2/12/2019 9:36:24 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.670","OfficeHumidity":"64"}]
2/12/2019 9:36:54 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.836","OfficeHumidity":"94"}]


string token = generateSasToken($"{server}/devices/{clientId}", password, "", new TimeSpan(0,5,0));
2/12/2019 9:40:52 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.46","OfficeHumidity":"92"}]
2/12/2019 9:41:22 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.443","OfficeHumidity":"62"}]
...
2/12/2019 9:50:55 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.742","OfficeHumidity":"95"}]


string token = generateSasToken($"{server}/devices/{clientId}", password, "", new TimeSpan(0,10,0));
approx 15min as only 30 sec resolution
1/12/2019 12:50:23 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.630","OfficeHumidity":"65"}]
1/12/2019 12:50:53 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.798","OfficeHumidity":"95"}]
...
1/12/2019 1:03:59 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.677","OfficeHumidity":"41"}]
1/12/2019 1:04:30 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.26","OfficeHumidity":"72"}]


string token = generateSasToken($"{server}/devices/{clientId}", password, "", new TimeSpan(0,10,0));
approx 15min as only 30 sec resolution
1/12/2019 1:09:30 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.106","OfficeHumidity":"72"}]
1/12/2019 1:10:00 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.463","OfficeHumidity":"42"}]
...
1/12/2019 1:23:35 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.366","OfficeHumidity":"77"}]
1/12/2019 1:24:05 PM> Device: [MQTTLoRa915MHz], Data:[{"OfficeTemperature":"22.537","OfficeHumidity":"47"}]

The dataset with the 5 minute expiry which remained connected for approximately 30 mins was hopefully a configuration issue.

The updated SAS Token code not uses ToUnixTimeSeconds to eliminate the scope for local vs. UTC issues.

      public static string generateSasToken(string resourceUri, string key, string policyName, TimeSpan timeToLive)
      {
         DateTimeOffset expiryDateTimeOffset = new DateTimeOffset(DateTime.UtcNow.Add(timeToLive));

         string expiryEpoch = expiryDateTimeOffset.ToUnixTimeSeconds().ToString();
         string stringToSign = WebUtility.UrlEncode(resourceUri) + "\n" + expiryEpoch;

         HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(key));
         string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));

         string token = $"SharedAccessSignature sr={WebUtility.UrlEncode(resourceUri)}&sig={WebUtility.UrlEncode(signature)}&se={expiryEpoch}";

         if (!String.IsNullOrEmpty(policyName))
         {
            token += "&skn=" + policyName;
         }

         return token;
      }

I need to test the expiry of my SAS Tokens some more especially with the client running on my development machine (NZT which is currently UTC+13) and in Azure (UTC timezone)

ThingsBoard with MQTTnet

As I’m testing my Message Queue Telemetry Transport(MQTT) LoRa gateway I’m building a proof of concept(PoC) .Net core console application for each IoT platform I would like to support.

This PoC was to confirm that I could connect to the ThingsBoard MQTT API then format topics and payloads correctly.

MQTTNet Console Client

The ThingsBoard MQTT broker, username, and client ID are the minimum command line options required with the CommandTopic optional.

   class Program
   {
      private static IMqttClient mqttClient = null;
      private static IMqttClientOptions mqttOptions = null;
      private static string server;
      private static string username;
      private static string clientId;
      private const string telemetryTopic = "v1/devices/me/telemetry";
      private static string commandTopic;

      static void Main(string[] args)
      {
         MqttFactory factory = new MqttFactory();
         mqttClient = factory.CreateMqttClient();

         if ((args.Length != 3) && (args.Length != 4))
         {
            Console.WriteLine("[MQTT Server] [UserName] [ClientID]");
            Console.WriteLine("[MQTT Server] [UserName] [ClientID] [CommandTopic]");
            Console.WriteLine("Press <enter> to exit");
            Console.ReadLine();
            return;
         }

         server =  args[0];
         username = args[1];
         clientId =  args[2];
         
         if (args.Length == 3)
         {
            Console.WriteLine($"MQTT Server:{server} ClientID:{clientId}");
         }

         if (args.Length == 4)
         {
            commandTopic = args[3];
            Console.WriteLine($"MQTT Server:{server} ClientID:{clientId} CommandTopic:{commandTopic}");
         }

         mqttOptions = new MqttClientOptionsBuilder()
            .WithTcpServer(server)
            .WithCredentials(username, "")
            .WithClientId(clientId)
            //.WithTls() blows up if this enabled, need to do more research on certificate config.
            .Build();

         mqttClient.UseDisconnectedHandler(new MqttClientDisconnectedHandlerDelegate(e => MqttClient_Disconnected(e)));
         mqttClient.UseApplicationMessageReceivedHandler(new MqttApplicationMessageReceivedHandlerDelegate(e => MqttClient_ApplicationMessageReceived(e)));
         mqttClient.ConnectAsync(mqttOptions).Wait();

         if (args.Length == 4)
         {
            mqttClient.SubscribeAsync(commandTopic, MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce).GetAwaiter().GetResult();
         }

         while (true)
         {
            JObject payloadJObject = new JObject();

            payloadJObject.Add("OfficeTemperature", "22." + DateTime.UtcNow.Millisecond.ToString());
            payloadJObject.Add("OfficeHumidity", (DateTime.UtcNow.Second + 40).ToString());

            string payload = JsonConvert.SerializeObject(payloadJObject);
            Console.WriteLine($"Topic:{telemetryTopic} Payload:{payload}");

            var message = new MqttApplicationMessageBuilder()
               .WithTopic(telemetryTopic)
               .WithPayload(payload)
               .WithAtLeastOnceQoS()
            .Build();

            Console.WriteLine("PublishAsync start");
            mqttClient.PublishAsync(message).Wait();
            Console.WriteLine("PublishAsync finish");

            Thread.Sleep(30100);
         }
      }

      private static void MqttClient_ApplicationMessageReceived(MqttApplicationMessageReceivedEventArgs e)
      {
         Console.WriteLine($"ClientId:{e.ClientId} Topic:{e.ApplicationMessage.Topic} Payload:{e.ApplicationMessage.ConvertPayloadToString()}");
      }

      private static async void MqttClient_Disconnected(MqttClientDisconnectedEventArgs e)
      {
         Debug.WriteLine("Disconnected");
         await Task.Delay(TimeSpan.FromSeconds(5));

         try
         {
            await mqttClient.ConnectAsync(mqttOptions);
         }
         catch (Exception ex)
         {
            Debug.WriteLine("Reconnect failed {0}", ex.Message);
         }
      }

The ThingsBoard device configuration was relatively easy with convenient buttons to copy the Device ID (Client ID in test client) and Access Token (UserName in test client). After looking at the source code for some of the other samples I figured out the ThingsBoard MQTT setup does not use the password field.

Device setup

The Device configuration form has a tab which displays the last telemetry which was useful for debugging. (I need to investigate the MQTT API support for claiming devices)

Device Telemetry display

To get telemetry data displayed on my dashboard I “added” it to my dashboard from the Entity configuration form. (I need to spend some more time watching the video tutorials to understand this process).

Device Entity View Setup

The dashboard designer had a number of “widgets” and what appeared to be the ability to add custom ones. Most forms also had “customer” option which appeared to be for multi-tenant support.

Selecting a dashboard graph widget
Configuring the y Axis Range for graph widget
Dashboard display home office Humidity & Temperature Information

Overall the initial configuration went smoothly after I figured out that the password was not required, and that Transport Layer Security(TLS) required some additional configuration.

Azure IoT Hub MQTT LoRa Field Gateway

Back in April I started working on an MQTT LoRa Field gateway which was going to support a selection of different Software as a service(SaaS) Internet of Things IoT) platforms.

I now have a working Azure IoT Hub plug-in (Azure IoT Central support as planned as well) with the first iteration focused on Device to Cloud (D2C) messaging. In a future iteration I will add Cloud to Device messaging(C2D).

My applications use a lightweight, easy to implemented protocol which is intended for hobbyist and educational use rather than commercial applications (I have been working on a more secure version as yet another side project)

I have a number of sample Arduino with Dragino LoRa Shield for Arduino, MakerFabs Maduino, Dragino LoRa Mini Dev, M2M Low power Node and Netduino with Elecrow LoRa RFM95 Shield etc. clients. These work with both my platform specific (Adafruit.IO, Azure IoT Hub/Central) gateways and protocol specific field gateways.

Azure IoT Hub Device Explorer Data Display

When the application is first started it creates a minimal configuration file which should be downloaded, the missing information filled out, then uploaded using the File explorer in the Windows device portal.

{
  "MQTTUserName": "YourIoTHubHub.azure-devices.net/MQTTLoRa915MHz/api-version=2018-06-30",
  "MQTTPassword": "SharedAccessSignature sr=YourIoTHubHub.azure-devices.net%2Fdevices%2FMQTTLoRa915MHz&sig=123456789012345678901234567890123456789012345%3D&se=1574673583",
  "MQTTClientID": "MQTTLoRa915MHz",
  "MQTTServer": "YourIoTHubHub.azure-devices.net",
  "Address": "LoRaIoT1",
  "Frequency": 915000000.0,
  "MessageHandlerAssembly": "Mqtt.IoTCore.FieldGateway.LoRa.AzureIoTHub",
  "PlatformSpecificConfiguration": ""
}

The application logs debugging information to the Windows 10 IoT Core ETW logging Microsoft-Windows-Diagnostics-LoggingChannel

MQTT LoRa Gateway with Azure IoT Hub plug-in

The message handler uploads all values in an inbound messages in one MQTT message.

namespace devMobile.Mqtt.IoTCore.FieldGateway
{
   using System;
   using System.Diagnostics;
   using System.Text;
   using Windows.Foundation.Diagnostics;

   using devMobile.IoT.Rfm9x;
   using MQTTnet;
   using MQTTnet.Client;
   using Newtonsoft.Json.Linq;
   using Newtonsoft.Json;

   public class MessageHandler : IMessageHandler
   {
      private LoggingChannel Logging { get; set; }
      private IMqttClient MqttClient { get; set; }
      private Rfm9XDevice Rfm9XDevice { get; set; }
      private string PlatformSpecificConfiguration { get; set; }

      void IMessageHandler.Initialise(LoggingChannel logging, IMqttClient mqttClient, Rfm9XDevice rfm9XDevice, string platformSpecificConfiguration)
      {
         LoggingFields processInitialiseLoggingFields = new LoggingFields();

         this.Logging = logging;
         this.MqttClient = mqttClient;
         this.Rfm9XDevice = rfm9XDevice;
         this.PlatformSpecificConfiguration = platformSpecificConfiguration;
      }

      async void IMessageHandler.Rfm9XOnReceive(Rfm9XDevice.OnDataReceivedEventArgs e)
      {
         LoggingFields processReceiveLoggingFields = new LoggingFields();
         char[] sensorReadingSeparators = { ',' };
         char[] sensorIdAndValueSeparators = { ' ' };

         processReceiveLoggingFields.AddString("PacketSNR", e.PacketSnr.ToString("F1"));
         processReceiveLoggingFields.AddInt32("PacketRSSI", e.PacketRssi);
         processReceiveLoggingFields.AddInt32("RSSI", e.Rssi);

         string addressBcdText = BitConverter.ToString(e.Address);
         processReceiveLoggingFields.AddInt32("DeviceAddressLength", e.Address.Length);
         processReceiveLoggingFields.AddString("DeviceAddressBCD", addressBcdText);

         string messageText;
         try
         {
            messageText = UTF8Encoding.UTF8.GetString(e.Data);
            processReceiveLoggingFields.AddString("MessageText", messageText);
         }
         catch (Exception ex)
         {
            processReceiveLoggingFields.AddString("Exception", ex.ToString());
            this.Logging.LogEvent("PayloadProcess failure converting payload to text", processReceiveLoggingFields, LoggingLevel.Warning);
            return;
         }

         // Chop up the CSV text
         string[] sensorReadings = messageText.Split(sensorReadingSeparators, StringSplitOptions.RemoveEmptyEntries);
         if (sensorReadings.Length < 1)
         {
            this.Logging.LogEvent("PayloadProcess payload contains no sensor readings", processReceiveLoggingFields, LoggingLevel.Warning);
            return;
         }

         JObject payloadJObject = new JObject();

         JObject feeds = new JObject();

         // Chop up each sensor read into an ID & value
         foreach (string sensorReading in sensorReadings)
         {
            string[] sensorIdAndValue = sensorReading.Split(sensorIdAndValueSeparators, StringSplitOptions.RemoveEmptyEntries);

            // Check that there is an id & value
            if (sensorIdAndValue.Length != 2)
            {
               this.Logging.LogEvent("PayloadProcess payload invalid format", processReceiveLoggingFields, LoggingLevel.Warning);
               return;
            }

            string sensorId = string.Concat(addressBcdText, sensorIdAndValue[0]);
            string value = sensorIdAndValue[1];

            feeds.Add(sensorId.ToLower(), value);
         }
         payloadJObject.Add("feeds", feeds);

         string topic = $"devices/{MqttClient.Options.ClientId}/messages/events/";

         try
         {
            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topic)
               .WithPayload(JsonConvert.SerializeObject(payloadJObject))
               .WithAtLeastOnceQoS()
               .Build();
            Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync start", DateTime.UtcNow);
            await MqttClient.PublishAsync(message);
            Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync finish", DateTime.UtcNow);

            this.Logging.LogEvent("PublishAsync Azure IoTHub payload", processReceiveLoggingFields, LoggingLevel.Information);
         }
         catch (Exception ex)
         {
            processReceiveLoggingFields.AddString("Exception", ex.ToString());
            this.Logging.LogEvent("PublishAsync Azure IoTHub payload", processReceiveLoggingFields, LoggingLevel.Error);
         }
      }

      void IMessageHandler.MqttApplicationMessageReceived(MqttApplicationMessageReceivedEventArgs e)
      {
         LoggingFields processReceiveLoggingFields = new LoggingFields();

         processReceiveLoggingFields.AddString("ClientId", e.ClientId);
#if DEBUG
         processReceiveLoggingFields.AddString("Payload", e.ApplicationMessage.ConvertPayloadToString());
#endif
         processReceiveLoggingFields.AddString("QualityOfServiceLevel", e.ApplicationMessage.QualityOfServiceLevel.ToString());
         processReceiveLoggingFields.AddBoolean("Retain", e.ApplicationMessage.Retain);
         processReceiveLoggingFields.AddString("Topic", e.ApplicationMessage.Topic);

         this.Logging.LogEvent("MqttApplicationMessageReceived topic not processed", processReceiveLoggingFields, LoggingLevel.Error);
      }

      void IMessageHandler.Rfm9xOnTransmit(Rfm9XDevice.OnDataTransmitedEventArgs e)
      {
      }
   }
}

The formatting of the username and generation of password are password are a bit awkward and will be fixed in a future refactoring. Along with regenerating the SAS connection token just before it is due to expire.