Azure IoT Hub SAS Tokens revisited

A long time ago I wrote a post about uploading telemetry data to an Azure Event Hub from a Netduino 3 Wifi using HTTPS. To send messages to the EventHub I had to create a valid SAS Token which took a surprising amount of effort because of the reduced text encoding/decoding and cryptographic functionality available in .NET Micro Framework v4.3 (NetMF)

// Create a SAS token for a specified scope. SAS tokens are described in http://msdn.microsoft.com/en-us/library/windowsazure/dn170477.aspx.
private static string CreateSasToken(string uri, string keyName, string key)
{
   // Set token lifetime to 20 minutes. When supplying a device with a token, you might want to use a longer expiration time.
   uint tokenExpirationTime = GetExpiry(20 * 60);
 
   string stringToSign = HttpUtility.UrlEncode(uri) + "\n" + tokenExpirationTime;
 
   var hmac = SHA.computeHMAC_SHA256(Encoding.UTF8.GetBytes(key), Encoding.UTF8.GetBytes(stringToSign));
   string signature = Convert.ToBase64String(hmac);
 
   signature = Base64NetMf42ToRfc4648(signature);
 
   string token = "SharedAccessSignature sr=" + HttpUtility.UrlEncode(uri) + "&sig=" + HttpUtility.UrlEncode(signature) + "&se=" + tokenExpirationTime.ToString() + "&skn=" + keyName;
 
   return token;
}
 
private static string Base64NetMf42ToRfc4648(string base64netMf)
{
   var base64Rfc = string.Empty;
 
   for (var i = 0; i < base64netMf.Length; i++)
   {
      if (base64netMf[i] == '!')
      {
         base64Rfc += '+';
      }
      else if (base64netMf[i] == '*')
      {
         base64Rfc += '/';
      }
      else
      {
         base64Rfc += base64netMf[i];
      }
   }
   return base64Rfc;
}
 
static uint GetExpiry(uint tokenLifetimeInSeconds)
{
   const long ticksPerSecond = 1000000000 / 100; // 1 tick = 100 nano seconds
 
   DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0);
   TimeSpan diff = DateTime.Now.ToUniversalTime() - origin;
 
   return ((uint)(diff.Ticks / ticksPerSecond)) + tokenLifetimeInSeconds;
}

Initially for testing my Azure MQTT Test Client I manually generated the SAS tokens using Azure Device Explorer but figured it would be better if the application generated them.

An initial search lead to this article about how to generate a SAS token for an Azure Event Hub in multiple languages. For my first attempt I “copied and paste” the code sample for C# (I also wasn’t certain what to put in the KeyName parameter) and it didn’t work.

private static string createToken(string resourceUri, string keyName, string key)
{
    TimeSpan sinceEpoch = DateTime.UtcNow - new DateTime(1970, 1, 1);
    var week = 60 * 60 * 24 * 7;
    var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + week);
    string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
    HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
    var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
    var sasToken = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}&skn={3}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry, keyName);
    return sasToken;
}

By comparing the Device Explorer and C# generated SAS keys I worked out the keyName parameter was unnecessary so I removed.

private static string createToken(string resourceUri, string key)
{
    TimeSpan sinceEpoch = DateTime.UtcNow - new DateTime(1970, 1, 1);
    var week = 60 * 60 * 24 * 7;
    var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + week);
    string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
    HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
    var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
    var sasToken = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry);
    return sasToken;
}

The shared SAS token now looked closer to what I was expecting but the MQTTNet ConnectAsync was failing with an authentication exception. After looking at the Device Explorer SAS Key code, my .NetMF implementation and the code for the IoT Hub SDK I noticed the encoding for the HMAC Key was different. Encoding.UTF8.GetBytes vs. Convert.FromBase64String.

 private static string createToken(string resourceUri,string key, TimeSpan ttl)
      {
         TimeSpan afterEpoch = DateTime.UtcNow.Add( ttl ) - new DateTime(1970, 1, 1);

         string expiry = afterEpoch.TotalSeconds.ToString("F0");
         string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
         HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(key));
         string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
         return  String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry);
      }

This approach appears to work reliably in my test harness.

MQTTnet client with new SAS Key Generator

User beware DIY Crypto often ends badly

Netduino LoRa Radio 433/868/915 MHz Payload Addressing client

This is a demo Netduino client (based on one of the examples in my RFM9XLoRaNetMF library) that uploads telemetry data to my Windows 10 IoT Core on Raspberry PI field gateway proof of concept(PoC).

Bill of materials (Prices Sep 2018)

//---------------------------------------------------------------------------------
// Copyright (c) 2017, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
namespace devMobile.IoT.Netduino.FieldGateway
{
   using System;
   using System.Text;
   using System.Threading;
   using Microsoft.SPOT;
   using Microsoft.SPOT.Hardware;
   using SecretLabs.NETMF.Hardware.Netduino;
   using devMobile.IoT.NetMF.ISM;
   using devMobile.NetMF.Sensor;

   class NetduinoClient
   {
      Rfm9XDevice rfm9XDevice;
      private readonly TimeSpan dueTime = new TimeSpan(0, 0, 15);
      private readonly TimeSpan periodTime = new TimeSpan(0, 0, 300);
      private readonly SiliconLabsSI7005 sensor = new SiliconLabsSI7005();
      private readonly OutputPort _led = new OutputPort(Pins.ONBOARD_LED, false);
      private readonly byte[] fieldGatewayAddress = Encoding.UTF8.GetBytes("LoRaIoT1");
      private readonly byte[] deviceAddress = Encoding.UTF8.GetBytes("Netduino1");

      public NetduinoClient()
      {
         rfm9XDevice = new Rfm9XDevice(Pins.GPIO_PIN_D10, Pins.GPIO_PIN_D9, Pins.GPIO_PIN_D2);
      }

      public void Run()
      {
         //rfm9XDevice.Initialise(frequency: 915000000, paBoost: true, rxPayloadCrcOn: true);
         rfm9XDevice.Initialise(frequency: 433000000, paBoost: true, rxPayloadCrcOn: true);
         rfm9XDevice.Receive(deviceAddress);

         rfm9XDevice.OnDataReceived += rfm9XDevice_OnDataReceived;
         rfm9XDevice.OnTransmit += rfm9XDevice_OnTransmit;

         Timer humidityAndtemperatureUpdates = new Timer(HumidityAndTemperatureTimerProc, null, dueTime, periodTime);

         Thread.Sleep(Timeout.Infinite);
      }

      private void HumidityAndTemperatureTimerProc(object state)
      {
         _led.Write(true);

         double humidity = sensor.Humidity();
         double temperature = sensor.Temperature();

         Debug.Print(DateTime.UtcNow.ToString("hh:mm:ss") + " H:" + humidity.ToString("F1") + " T:" + temperature.ToString("F1"));

         rfm9XDevice.Send(fieldGatewayAddress, Encoding.UTF8.GetBytes( "t " + temperature.ToString("F1") + ",H " + humidity.ToString("F0")));

         _led.Write(true);
      }

      void rfm9XDevice_OnTransmit()
      {
         Debug.Print("Transmit-Done");
         _led.Write(false);
      }

      void rfm9XDevice_OnDataReceived(byte[] address, float packetSnr, int packetRssi, int rssi, byte[] data)
      {
         try
         {
            string messageText = new string(UTF8Encoding.UTF8.GetChars(data));
            string addressText = new string(UTF8Encoding.UTF8.GetChars(address));

            Debug.Print(DateTime.UtcNow.ToString("HH:MM:ss") + "-Rfm9X PacketSnr " + packetSnr.ToString("F1") + " Packet RSSI " + packetRssi + "dBm RSSI " + rssi + "dBm = " + data.Length + " byte message " + @"""" + messageText + @"""");
         }
         catch (Exception ex)
         {
            Debug.Print(ex.Message);
         }
      }
   }
}

The code is available on GitHub
FieldGatewayNetduinoLoRaElecrow915
Elecrow shield
FieldGatewayNetduinoLoRaDragino915
Dragino shield
FieldGatewayNetduinLoRaMakerFabs433
MakerFabs shield
Net Micro Framework debug output from device

The thread '' (0x2) has exited with code 0 (0x0).
12:00:18 H:96.9 T:19.6
Transmit-Done
12:05:17 H:95.1 T:20.1
Transmit-Done

.Net Framework debug output Field Gateway

The thread 0x1550 has exited with code 0 (0x0).
21:21:49-RX From Netduino1 PacketSnr 9.5 Packet RSSI -40dBm RSSI -107dBm = 11 byte message "t 19.6,H 97"
 Sensor Netduino1t Value 19.6
 Sensor Netduino1H Value 97
 AzureIoTHubClient SendEventAsync start
 AzureIoTHubClient SendEventAsync finish
...
21:26:49-RX From Netduino1 PacketSnr 9.5 Packet RSSI -33dBm RSSI -103dBm = 11 byte message "t 20.1,H 95"
 Sensor Netduino1t Value 20.1
 Sensor Netduino1H Value 95
 AzureIoTHubClient SendEventAsync start
 AzureIoTHubClient SendEventAsync finish
The thread 0xfbc has exited with code 0 (0x0).

Then in my Azure IoT Hub

AzureIOTHubExplorerScreenGrab20180917

Azure IoT Hubs LoRa Windows 10 IoT Core Field Gateway

This project is now live on github.com, sample Arduino with Dragino LoRa Shield for Arduino, MakerFabs Maduino, Dragino LoRa Mini Dev, M2M Low power Node and Netduino with Elecrow LoRa RFM95 Shield clients uploaded in the next couple of days.

AzureIOTHubExplorerScreenGrab20180912

The bare minimum configuration is

{
  "AzureIoTHubDeviceConnectionString": "HostName=qwertyuiop.azure-devices.net;DeviceId=LoRaGateway;SharedAccessKey=1234567890qwertyuiop987654321qwertyuiop1234g=",
  "AzureIoTHubTransportType": "Amqp",
  "SensorIDIsDeviceIDSensorID": true,
  "Address": "LoRaIoT1",
  "Frequency": 915000000.0
}

So far battery life and wireless communications range for the Arduino clients is looking pretty good. CRC presence checking and validation is turned so have a look at one of the sample clients.

ArduinoUnoR3DraginoLoRa
It took a bit longer than expected as upgrading to the latest version (v1.18.0 as at 12 Sep 2018) of Microsoft.Azure.Devices.Client (from 1.6.3) broke my field gateway with timeouts and exceptions.

I’ll be doing some more testing over the next couple of weeks so it is a work in progress.

AdaFruit.IO LoRa Windows 10 IoT Core Field Gateway

This project is now live on github.com, sample Arduino with Dragino LoRa Shield for Arduino, MakerFabs Maduino, Dragino LoRa Mini Dev, M2M Low power Node and Netduino with Elecrow LoRa RFM95 Shield clients uploaded in the next couple of days.

AdaFruit.IO.LoRaScreenShot
While building this AdaFruit.IO LoRa field gateway, and sample clients I revisited my RFM9XLoRa-Net library a couple of times adding functionality and renaming constants to make it more consistent. I made many of the default values public so they could be used in the field gateway config file.
The bare minimum configuration is

{
“AdaFruitIOUserName”: “——“,
“AdaFruitIOApiKey”: “——“,
“AdaFruitIOGroupName”: “——”
“Address”: “——“,
“Frequency”: 915000000.0
}

So far battery life and wireless communications range for the Arduino clients is looking pretty good.

ArduinoUnoR3DraginoLoRa

RFM9X.NetMF Payload Addressing

I have extended theĀ NetMF sample application and library to show how the conditional compilation directive ADDRESSED_MESSAGES_PAYLOAD controls the configuration.

When the application is started the RFM9X is in sleep mode, then when the Receive method is called the device is set to ReceiveContinuous.

//---------------------------------------------------------------------------------
// Copyright (c) August 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.NetMF.Rfm9X.Client
{
   using System;
   using System.Text;
   using System.Threading;
   using devMobile.IoT.NetMF.ISM;
   using Microsoft.SPOT;
   using SecretLabs.NETMF.Hardware.Netduino;

   public class Program
   {
      public static void Main()
      {
         Rfm9XDevice rfm9XDevice = new Rfm9XDevice(Pins.GPIO_PIN_D10, Pins.GPIO_PIN_D9, Pins.GPIO_PIN_D2);
         byte MessageCount = Byte.MinValue;

         rfm9XDevice.Initialise( frequency:915000000, paBoost: true, rxPayloadCrcOn: true);
#if ADDRESSED_MESSAGES
         rfm9XDevice.Receive(Encoding.UTF8.GetBytes("Netduino"));
#else
         rfm9XDevice.Receive();
#endif
         rfm9XDevice.OnDataReceived += rfm9XDevice_OnDataReceived;
         rfm9XDevice.OnTransmit += rfm9XDevice_OnTransmit;

         while (true)
         {
            string messageText = "Hello NetMF LoRa! " + MessageCount.ToString();
            MessageCount += 1;
            byte[] messageBytes = UTF8Encoding.UTF8.GetBytes(messageText);
            Debug.Print("Sending " + messageBytes.Length + " bytes message " + messageText);

#if ADDRESSED_MESSAGES
            rfm9XDevice.Send(UTF8Encoding.UTF8.GetBytes("LoRaIoT1"), messageBytes);
#else
            rfm9XDevice.Send(messageBytes);
#endif
            Thread.Sleep(10000);
         }
      }

      static void rfm9XDevice_OnTransmit()
      {
         Debug.Print("Transmit-Done");
      }

#if ADDRESSED_MESSAGES
      static void rfm9XDevice_OnDataReceived(byte[] address, float packetSnr, int packetRssi, int rssi, byte[] data)
#else
      static void rfm9XDevice_OnDataReceived(float packetSnr, int packetRssi, int rssi,  byte[] data)
#endif
      {
         try
         {
            string messageText = new string(UTF8Encoding.UTF8.GetChars(data));
#if ADDRESSED_MESSAGES
            string addressText = new string(UTF8Encoding.UTF8.GetChars(address));

            Debug.Print(DateTime.UtcNow.ToString("HH:MM:ss") + "-From " + addressText + " PacketSnr " + packetSnr.ToString("F1") + " Packet RSSI " + packetRssi + "dBm RSSI " + rssi + "dBm = " + data.Length + " byte message " + @"""" + messageText + @"""") ;
#else
            Debug.Print(DateTime.UtcNow.ToString("HH:MM:ss") + "-Rfm9X PacketSnr " + packetSnr.ToString("F1") + " Packet RSSI " + packetRssi + "dBm RSSI " + rssi + "dBm = " + data.Length + " byte message " + @"""" + messageText + @"""") ;
#endif
         }
         catch (Exception ex)
         {
            Debug.Print(ex.Message);
         }
      }
   }
}

namespace System.Diagnostics
{
   public enum DebuggerBrowsableState
   {
      Never = 0,
      Collapsed = 2,
      RootHidden = 3
   }
}

The Netduino client “plays nicely” with my Windows 10 IoT Core on Raspberry PI field gateway proof of concept(PoC).

The Semech SX127X datasheet describes how addressing can be implemented using interrupts which I will have a look at soon.

Library needs further testing and I’m working on a sample Arduino application.

RFM9X.IoTCore Payload Addressing

The reason for RFM9XLoRaNet was so that I could build a field gateway to upload telemetry data from “cheap n cheerful” *duino devices to Azure IoT Hubs and AdaFruit.IO.

I have extended the Windows10IoTCore sample application and library to show how the conditional compilation directive ADDRESSED_MESSAGES_PAYLOAD controls the configuration.

When the application is started the RFM9X is in sleep mode, then when the Receive method is called the device is set to ReceiveContinuous.

public void Run(IBackgroundTaskInstance taskInstance)
{
   rfm9XDevice.Initialise(915000000.0, paBoost: true, rxPayloadCrcOn : true);

#if DEBUG
   rfm9XDevice.RegisterDump();
#endif

#if ADDRESSED_MESSAGES_PAYLOAD
   rfm9XDevice.OnReceive += Rfm9XDevice_OnReceive;
   rfm9XDevice.Receive(UTF8Encoding.UTF8.GetBytes(Environment.MachineName));
#else
   rfm9XDevice.Receive();
#endif
   rfm9XDevice.OnTransmit += Rfm9XDevice_OnTransmit;

   Task.Delay(10000).Wait();

   while (true)
   {
      string messageText = string.Format("Hello from {0} ! {1}", Environment.MachineName, NessageCount);
      MessageCount -= 1;

      byte[] messageBytes = UTF8Encoding.UTF8.GetBytes(messageText);
      Debug.WriteLine("{0:HH:mm:ss}-TX {1} byte message {2}", DateTime.Now, messageBytes.Length, messageText);
#if ADDRESSED_MESSAGES_PAYLOAD
      this.rfm9XDevice.Send(UTF8Encoding.UTF8.GetBytes("AddressGoesHere"), messageBytes);
#else
      this.rfm9XDevice.Send(messageBytes);
#endif
      Task.Delay(10000).Wait();
   }
}

On receipt of a message, the message is parsed and the to/from addresses and payload extracted (ADDRESSED_MESSAGES defined) or passed to the client application for processing.

private void Rfm9XDevice_OnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e)
{
   try
   {
      string messageText = UTF8Encoding.UTF8.GetString(e.Data);

#if ADDRESSED_MESSAGES_PAYLOAD
      string addressText = UTF8Encoding.UTF8.GetString(e.Address);

      Debug.WriteLine(@"{0:HH:mm:ss}-RX From {1} PacketSnr {2:0.0} Packet RSSI {3}dBm RSSI {4}dBm = {5} byte message ""{6}""", DateTime.Now, addressText, e.PacketSnr, e.PacketRssi, e.Rssi, e.Data.Length, messageText);
#else
      Debug.WriteLine(@"{0:HH:mm:ss}-RX PacketSnr {1:0.0} Packet RSSI {2}dBm RSSI {3}dBm = {4} byte message ""{5}""", DateTime.Now, e.PacketSnr, e.PacketRssi, e.Rssi, e.Data.Length, messageText);
#endif
   }
   catch (Exception ex)
   {
      Debug.WriteLine(ex.Message);
   }
}

The addressing implementation needs further testing and I’m building sample .NetMF and *duino clients.

Rfm9xLoRaDevice NetMF Payload CRCs

To ensure I was only handling messages with valid contents I added code to appended a cyclic redundancy check(CRC) onto outbound messages and validate the CRC on inbound messages.

First step was to update the initialise method parameter list (the parameter list is huge but for most scenarios the defaults are fine)

public void Initialise(RegOpModeMode regOpModeAfterInitialise, // RegOpMode
         double frequency = FrequencyDefault, // RegFrMsb, RegFrMid, RegFrLsb
         bool rxDoneignoreIfCrcMissing = true, bool rxDoneignoreIfCrcInvalid = true,
         bool paBoost = false, byte maxPower = RegPAConfigMaxPowerDefault, byte outputPower = RegPAConfigOutputPowerDefault, // RegPaConfig
         bool ocpOn = true, byte ocpTrim = RegOcpOcpTrimDefault, // RegOcp
         RegLnaLnaGain lnaGain = LnaGainDefault, bool lnaBoost = false, // RegLna
         RegModemConfigBandwidth bandwidth = RegModemConfigBandwidthDefault, RegModemConfigCodingRate codingRate = RegModemConfigCodingRateDefault, RegModemConfigImplicitHeaderModeOn implicitHeaderModeOn = RegModemConfigImplicitHeaderModeOnDefault, //RegModemConfig1
         RegModemConfig2SpreadingFactor spreadingFactor = RegModemConfig2SpreadingFactorDefault, bool txContinuousMode = false, bool rxPayloadCrcOn = false,
         ushort symbolTimeout = SymbolTimeoutDefault,
         ushort preambleLength = PreambleLengthDefault,
         byte payloadLength = PayloadLengthDefault,
         byte payloadMaxLength = PayloadMaxLengthDefault,
         byte freqHoppingPeriod = FreqHoppingPeriodDefault,
         bool lowDataRateOptimize = false, bool agcAutoOn = false,
         byte ppmCorrection = ppmCorrectionDefault,
         RegDetectOptimizeDectionOptimize detectionOptimize = RegDetectOptimizeDectionOptimizeDefault,
         bool invertIQ = false,
         RegisterDetectionThreshold detectionThreshold = RegisterDetectionThresholdDefault,
         byte syncWord = RegSyncWordDefault)

The rxPayloadCrcOn needs to be set to True for outbound messages to have a CRC.

Then in the RxDone interrupt handler the CRC is checked (regHopChannel & regIrqFlagsMask) if this feature is enabled. Any messages with missing\invalid CRCs will currently be silently discarded and I’m not certain this is a good idea.

 // Check to see if payload has CRC
         if (RxDoneIgnoreIfCrcMissing)
         {
            byte regHopChannel = this.Rfm9XLoraModem.ReadByte((byte)Registers.RegHopChannel);
            if ((regHopChannel & (byte)RegHopChannelFlags.CrcOnPayload) != (byte)RegHopChannelFlags.CrcOnPayload)
            {
               return;
            }
         }

         // Check to see if payload CRC is valid
         if (RxDoneIgnoreIfCrcInvalid)
         {
            if (((byte)IrqFlags & (byte)RegIrqFlagsMask.PayLoadCrcErrorMask) == (byte)RegIrqFlagsMask.PayLoadCrcErrorMask)
            {
               return;
            }
         }

The conversion of the payload from an array of bytes to a string for display stopped failing with an exception. When I had a number of clients running up to 10% of the messages were getting corrupted.