Azure IoT Hub SAS Tokens revisited

A long time ago I wrote a post about uploading telemetry data to an Azure Event Hub from a Netduino 3 Wifi using HTTPS. To send messages to the EventHub I had to create a valid SAS Token which took a surprising amount of effort because of the reduced text encoding/decoding and cryptographic functionality available in .NET Micro Framework v4.3 (NetMF)

// Create a SAS token for a specified scope. SAS tokens are described in http://msdn.microsoft.com/en-us/library/windowsazure/dn170477.aspx.
private static string CreateSasToken(string uri, string keyName, string key)
{
   // Set token lifetime to 20 minutes. When supplying a device with a token, you might want to use a longer expiration time.
   uint tokenExpirationTime = GetExpiry(20 * 60);
 
   string stringToSign = HttpUtility.UrlEncode(uri) + "\n" + tokenExpirationTime;
 
   var hmac = SHA.computeHMAC_SHA256(Encoding.UTF8.GetBytes(key), Encoding.UTF8.GetBytes(stringToSign));
   string signature = Convert.ToBase64String(hmac);
 
   signature = Base64NetMf42ToRfc4648(signature);
 
   string token = "SharedAccessSignature sr=" + HttpUtility.UrlEncode(uri) + "&sig=" + HttpUtility.UrlEncode(signature) + "&se=" + tokenExpirationTime.ToString() + "&skn=" + keyName;
 
   return token;
}
 
private static string Base64NetMf42ToRfc4648(string base64netMf)
{
   var base64Rfc = string.Empty;
 
   for (var i = 0; i < base64netMf.Length; i++)
   {
      if (base64netMf[i] == '!')
      {
         base64Rfc += '+';
      }
      else if (base64netMf[i] == '*')
      {
         base64Rfc += '/';
      }
      else
      {
         base64Rfc += base64netMf[i];
      }
   }
   return base64Rfc;
}
 
static uint GetExpiry(uint tokenLifetimeInSeconds)
{
   const long ticksPerSecond = 1000000000 / 100; // 1 tick = 100 nano seconds
 
   DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0);
   TimeSpan diff = DateTime.Now.ToUniversalTime() - origin;
 
   return ((uint)(diff.Ticks / ticksPerSecond)) + tokenLifetimeInSeconds;
}

Initially for testing my Azure MQTT Test Client I manually generated the SAS tokens using Azure Device Explorer but figured it would be better if the application generated them.

An initial search lead to this article about how to generate a SAS token for an Azure Event Hub in multiple languages. For my first attempt I “copied and paste” the code sample for C# (I also wasn’t certain what to put in the KeyName parameter) and it didn’t work.

private static string createToken(string resourceUri, string keyName, string key)
{
    TimeSpan sinceEpoch = DateTime.UtcNow - new DateTime(1970, 1, 1);
    var week = 60 * 60 * 24 * 7;
    var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + week);
    string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
    HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
    var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
    var sasToken = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}&skn={3}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry, keyName);
    return sasToken;
}

By comparing the Device Explorer and C# generated SAS keys I worked out the keyName parameter was unnecessary so I removed.

private static string createToken(string resourceUri, string key)
{
    TimeSpan sinceEpoch = DateTime.UtcNow - new DateTime(1970, 1, 1);
    var week = 60 * 60 * 24 * 7;
    var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + week);
    string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
    HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
    var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
    var sasToken = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry);
    return sasToken;
}

The shared SAS token now looked closer to what I was expecting but the MQTTNet ConnectAsync was failing with an authentication exception. After looking at the Device Explorer SAS Key code, my .NetMF implementation and the code for the IoT Hub SDK I noticed the encoding for the HMAC Key was different. Encoding.UTF8.GetBytes vs. Convert.FromBase64String.

 private static string createToken(string resourceUri,string key, TimeSpan ttl)
      {
         TimeSpan afterEpoch = DateTime.UtcNow.Add( ttl ) - new DateTime(1970, 1, 1);

         string expiry = afterEpoch.TotalSeconds.ToString("F0");
         string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
         HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(key));
         string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
         return  String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry);
      }

This approach appears to work reliably in my test harness.

MQTTnet client with new SAS Key Generator

User beware DIY Crypto often ends badly

AdaFruit IO basic Netduino HTTP client

I use Netduino devices for teaching and my students often build projects which need a cloud based service like AdaFruit.IO to capture, store and display their sensor data.

My Proof of Concept (PoC) which uses a slightly modified version of the AdaFruit.IO basic desktop HTTP client code has been running on several Netduino 2 Plus, Netduino 3 Ethernet and Netduino 3 Wifi devices for the last couple of days and looks pretty robust.

The Netduino 3 Wifi device also supports https for improved security and privacy. They also make great field gateways as they can run off solar/battery power.

N2PN3WDashBoard

The devices have been uploading temperature and humidity measurements from a Silicon labs Si7005 sensor. (Outside sensor suffering from sunstrike)

N3WifiTemperatureAndHumiditySensor

program.cs

*

Copyright ® 2017 December devMobile Software, All Rights Reserved

THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR
PURPOSE.

http://www.devmobile.co.nz

*/
using System;
using System.Net;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using Microsoft.SPOT.Net.NetworkInformation;
using SecretLabs.NETMF.Hardware.Netduino;
using devMobile.NetMF.Sensor;
using devMobile.IoT.NetMF;

namespace devMobile.IoT.AdaFruitIO.NetMF.Client
{
   public class Program
   {
      private const string adaFruitIOApiBaseUrl = @"https://IO.adafruit.com/api/v2/";
      private const string group = "netduino3";
      private const string temperatureFeedKey = "t";
      private const string humidityFeedKey = "h";
      private const string adaFruitUserName = "YourUserName";
      private const string adaFruitIOApiKey = "YourAPIKey";
      private static readonly TimeSpan timerDueAfter = new TimeSpan(0, 0, 15);
      private static readonly TimeSpan timerPeriod = new TimeSpan(0, 0, 30);
      private static OutputPort led = new OutputPort(Pins.ONBOARD_LED, false);
      private static SiliconLabsSI7005 sensor = new SiliconLabsSI7005();
      private static AdaFruitIoClient adaFruitIoClient = new AdaFruitIoClient(adaFruitUserName, adaFruitIOApiKey, adaFruitIOApiBaseUrl);

      public static void Main()
      {
         // Wait for Network address if DHCP
         NetworkInterface networkInterface = NetworkInterface.GetAllNetworkInterfaces()[0];
         if (networkInterface.IsDhcpEnabled)
         {
            Debug.Print(" Waiting for DHCP IP address");

            while (NetworkInterface.GetAllNetworkInterfaces()[0].IPAddress == IPAddress.Any.ToString())
            {
               Debug.Print(" .");
               led.Write(!led.Read());
               Thread.Sleep(250);
            }
            led.Write(false);
         }

         // Display network config for debugging
         Debug.Print("Network configuration");
         Debug.Print(" Network interface type : " + networkInterface.NetworkInterfaceType.ToString());
         Debug.Print(" MAC Address : " + BytesToHexString(networkInterface.PhysicalAddress));
         Debug.Print(" DHCP enabled : " + networkInterface.IsDhcpEnabled.ToString());
         Debug.Print(" Dynamic DNS enabled : " + networkInterface.IsDynamicDnsEnabled.ToString());
         Debug.Print(" IP Address : " + networkInterface.IPAddress.ToString());
         Debug.Print(" Subnet Mask : " + networkInterface.SubnetMask.ToString());
         Debug.Print(" Gateway : " + networkInterface.GatewayAddress.ToString());

         foreach (string dnsAddress in networkInterface.DnsAddresses)
         {
            Debug.Print(" DNS Server : " + dnsAddress.ToString());
         }

         Timer humidityAndtemperatureUpdates = new Timer(HumidityAndTemperatureTimerProc, null, timerDueAfter, timerPeriod);

         Thread.Sleep(Timeout.Infinite);
      }

      static private void HumidityAndTemperatureTimerProc(object state)
      {
         led.Write(true);

         try
         {
            double humidity = sensor.Humidity();

            Debug.Print(" Humidity " + humidity.ToString("F0") + "%");
            adaFruitIoClient.FeedUpdate(group, humidityFeedKey, humidity.ToString("F0"));
         }
         catch (Exception ex)
         {
            Debug.Print("Humidifty read+update failed " + ex.Message);

            return;
         }

         try
         {
            double temperature = sensor.Temperature();

            Debug.Print(" Temperature " + temperature.ToString("F1") + "°C");
            adaFruitIoClient.FeedUpdate(group, temperatureFeedKey, temperature.ToString("F1"));
         }
         catch (Exception ex)
         {
            Debug.Print("Temperature read+update failed " + ex.Message);

            return;
         }

         led.Write(false);
      }

      private static string BytesToHexString(byte[] bytes)
      {
         string hexString = string.Empty;

         // Create a character array for hexidecimal conversion.
         const string hexChars = "0123456789ABCDEF";

         // Loop through the bytes.
         for (byte b = 0; b < bytes.Length; b++)          {             if (b > 0)
               hexString += "-";

            // Grab the top 4 bits and append the hex equivalent to the return string.
            hexString += hexChars[bytes[b] >> 4];

            // Mask off the upper 4 bits to get the rest of it.
            hexString += hexChars[bytes[b] & 0x0F];
         }

         return hexString;
      }
   }
}

AdaFruit.IO client.cs, handles feed groups and individual feeds

/*

Copyright ® 2017 December devMobile Software, All Rights Reserved

THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR
PURPOSE.

http://www.devmobile.co.nz

*/
using System;
using System.IO;
using System.Net;
using System.Text;
using Microsoft.SPOT;

namespace devMobile.IoT.NetMF
{
   public class AdaFruitIoClient
   {
      private const string apiBaseUrlDefault = @"http://IO.adafruit.com/api/v2/";
      private string apiBaseUrl = "";
      private string userName = "";
      private string apiKey = "";
      private int httpRequestTimeoutmSec;
      private int httpRequestReadWriteTimeoutmSec;

      public AdaFruitIoClient(string userName, string apiKey, string apiBaseUrl = apiBaseUrlDefault, int httpRequestTimeoutmSec = 2500, int httpRequestReadWriteTimeoutmSec = 5000)
      {
         this.apiBaseUrl = apiBaseUrl;
         this.userName = userName;
         this.apiKey = apiKey;
         this.httpRequestReadWriteTimeoutmSec = httpRequestReadWriteTimeoutmSec;
         this.httpRequestTimeoutmSec = httpRequestTimeoutmSec;
      }

      public void FeedUpdate(string group, string feedKey, string value)
      {
         string feedUrl;

         if (group.Trim() == string.Empty)
         {
            feedUrl = apiBaseUrl + userName + @"/feeds/" + feedKey + @"/data";
         }
         else
         {
            feedUrl = apiBaseUrl + userName + @"/feeds/" + group.Trim() + "." + feedKey + @"/data";
         }

         HttpWebRequest request = (HttpWebRequest)WebRequest.Create(feedUrl);
         {
            string payload = @"{""value"": """ + value + @"""}";
            byte[] buffer = Encoding.UTF8.GetBytes(payload);

            DateTime httpRequestedStartedAtUtc = DateTime.UtcNow;

            request.Method = "POST";
            request.ContentLength = buffer.Length;
            request.ContentType = @"application/json";
            request.Headers.Add("X-AIO-Key", apiKey);
            request.KeepAlive = false;
            request.Timeout = this.httpRequestTimeoutmSec;
            request.ReadWriteTimeout = this.httpRequestReadWriteTimeoutmSec;

            using (Stream stream = request.GetRequestStream())
            {
               stream.Write(buffer, 0, buffer.Length);
            }

            using (var response = (HttpWebResponse)request.GetResponse())
            {
               Debug.Print(" Status: " + response.StatusCode + " : " + response.StatusDescription);
            }

            TimeSpan duration = DateTime.UtcNow - httpRequestedStartedAtUtc;
            Debug.Print(" Duration: " + duration.ToString());
         }
      }
   }
}

Bill of materials for PoC

Netduino 3 Wifi Queued Azure Event Hub Field Gateway V1.0

My ADSL connection had been a bit flaky which had meant I had lost some sensor data with my initial Azure Event Hub gateway. In attempt make the solution more robust this version of the gateway queues unsent messages using the on-board MicroSD card support.

The code assumes that a file move is an “atomic operation”, so it streams the events received from the devices into a temporary directory (configurable) then moves them to the upload directory (configurable).

This code is proof of concept and needs to be soak tested, improved error handling and some additional multi threading locking added plus the magic constants refactored.

This code is called in the nRF24 receive messages handler

private void OnReceive(byte[] data)
{
   activityLed.Write(!activityLed.Read());

   // Ensure that we have a payload
   if (data.Length < 1 )
   {
      Debug.Print( "ERROR - Message has no payload" ) ;
      return ;
   }

   string message = new String(Encoding.UTF8.GetChars(data));
   Debug.Print("+" + DateTime.UtcNow.ToString("HH:mm:ss") + " L=" + data.Length + " M=" + message);

   string filename = DateTime.UtcNow.ToString("yyyyMMddhhmmssff") + ".txt";

   string tempDirectory = Path.Combine("\\sd", "temp");
   string tempFilePath = Path.Combine(tempDirectory, filename);

   string queueDirectory = Path.Combine("\\sd", "data");
   string queueFilePath = Path.Combine(queueDirectory, filename);

   File.WriteAllBytes(tempFilePath, data);

   File.Move(tempFilePath, queueFilePath);

   new Microsoft.SPOT.IO.VolumeInfo("\\sd").FlushAll();
}

A timer initiates the upload process which uses the AMQPNetlite library

bool UploadInProgress = false;

      
void uploaderCallback(object state)
{
   Debug.Print("uploaderCallback - start");

   if (UploadInProgress)
   {
      return;
   }
   UploadInProgress = true;

   string[] eventFilesToSend = Directory.GetFiles(Path.Combine("\\sd", "data")) ;

   if ( eventFilesToSend.Length == 0 )
   {
      Debug.Print("uploaderCallback - no files");
      UploadInProgress = false;
      return ;
   }

   try
   {
      Debug.Print("uploaderCallback - Connect");
      Connection connection = new Connection(new Address(serviceBusHost, serviceBusPort, serviceBusSasKeyName, serviceBusSasKey));

      Session session = new Session(connection);

      SenderLink sender = new SenderLink(session, "send-link", eventHubName);

      for (int index = 0; index < System.Math.Min(eventUploadBatchSizeMaximum, eventFilesToSend.Length); index++)
      {
         string eventFile = eventFilesToSend[ index ] ;

         Debug.Print("-" + DateTime.UtcNow.ToString("HH:mm:ss") + " " + eventFile ); ;

         Message message = new Message()
         {
            BodySection = new Data()
            {
               Binary = File.ReadAllBytes(eventFile),
            },
         ApplicationProperties = new Amqp.Framing.ApplicationProperties(),
         };

         FileInfo fileInfo = new FileInfo(eventFile);

         message.ApplicationProperties["AcquiredAtUtc"] = fileInfo.CreationTimeUtc;
         message.ApplicationProperties["UploadedAtUtc"] = DateTime.UtcNow;
         message.ApplicationProperties["GatewayId"] = gatewayId;
         message.ApplicationProperties["DeviceId"] = deviceId;
         message.ApplicationProperties["EventId"] = Guid.NewGuid();

         sender.Send(message);

         File.Delete(eventFile);

         new Microsoft.SPOT.IO.VolumeInfo("\\sd").FlushAll();
      }

      sender.Close();
      session.Close();
      connection.Close();
   }
   catch (Exception ex)
   {
      Debug.Print("ERROR: Upload failed with error: " + ex.Message);
   }
   finally
   {
      Debug.Print("uploaderCallback - finally");
      UploadInProgress = false;
   }
}

The timer period and number of files uploaded in each batch is configurable. I need to test the application to see how it handles power outages and MicroSD card corruption. The source is Netduino NRF24L01 AMQPNetLite Queued Azure EventHub Gatewaywith all the usual caveats.

This project wouldn’t have been possible without

Netduino 3 AnalogInput read rates

At CodeClub some of the students build a power consumption meter and as part of that project we measure the AnalogInput sample rates to check they are sufficient for our application.

Earlier this term when we measured the sampling rates in a CodeClub session we had a mix of Netduino 2 and Netduino 3 devices and some of the results differed from my previous observations. I used the same code on all the devices

int value;
AnalogInput x1 = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);
stopwatch.Start();
for (int i = 0; i < SampleCount; i++)
{
value = x1.ReadRaw();
}
stopwatch.Stop();

Netduino Plus 2

Duration = 2081 mSec 48053/sec
Duration = 2082 mSec 48030/sec
Duration = 2081 mSec 48053/sec
Duration = 2081 mSec 48053/sec
Duration = 2082 mSec 48030/sec
Duration = 2081 mSec 48053/sec
Duration = 2081 mSec 48053/sec
Duration = 2081 mSec 48053/sec
Duration = 2081 mSec 48053/sec
Duration = 2081 mSec 48053/sec

Netduino 3

Duration = 2071 mSec 48285/sec
Duration = 2069 mSec 48332/sec
Duration = 2070 mSec 48309/sec
Duration = 2071 mSec 48285/sec
Duration = 2071 mSec 48285/sec
Duration = 2070 mSec 48309/sec
Duration = 2070 mSec 48309/sec
Duration = 2071 mSec 48285/sec
Duration = 2071 mSec 48285/sec
Duration = 2071 mSec 48285/sec

Netduino 3 Ethernet
Duration = 2136 mSec 46816/sec
Duration = 2137 mSec 46794/sec
Duration = 2136 mSec 46816/sec
Duration = 2135 mSec 46838/sec
Duration = 2135 mSec 46838/sec
Duration = 2137 mSec 46794/sec
Duration = 2137 mSec 46794/sec
Duration = 2135 mSec 46838/sec
Duration = 2136 mSec 46816/sec
Duration = 2135 mSec 46838/sec

Netduino 3 Wifii
Duration = 3902 mSec 25627/sec
Duration = 3901 mSec 25634/sec
Duration = 3902 mSec 25627/sec
Duration = 3902 mSec 25627/sec
Duration = 3901 mSec 25634/sec
Duration = 3903 mSec 25621/sec
Duration = 3903 mSec 25621/sec
Duration = 3902 mSec 25627/sec
Duration = 3902 mSec 25627/sec
Duration = 3903 mSec 25621/sec

The results for the Netduino 3 & Netduino 3 Ethernet were comparable with the Netduino Plus 2 in my earlier post. The reduction in the sampling rate of the Netduino 3 Wifi warrants some further investigation.

Netduino 3 Wifi pollution Sensor Part 2

In a previous post I had started building a driver for the Seeedstudio Grove Dust Sensor. It was a proof of concept and it didn’t handle some edge cases well.

While building the pollution monitor with a student we started by simulating the negative occupancy of the Shinyei PPD42NJ Particle sensor with the Netduino’s on-board button. This worked and reduced initial complexity. But it also made it harder to simulate the button being pressed as the program launches (the on-board button is also the reset button), or simulate if the button was pressed at the start or end of the period.

Dust sensor simulation with button

Netduino 3 Wifi Test Harness

The first sample code processes button press interrupts and displays the values of the data1 & data2 parameters

public class Program
{
   public static void Main()
   {
      InterruptPort button = new InterruptPort(Pins.GPIO_PIN_D5, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);
      button.OnInterrupt += button_OnInterrupt;

      Thread.Sleep(Timeout.Infinite);
   }

   static void button_OnInterrupt(uint data1, uint data2, DateTime time)
   {
      Debug.Print(time.ToString("hh:mm:ss.fff") + " data1 =" + data1.ToString() + " data2 = " + data2.ToString());
   }
}

Using the debugging output from this application we worked out that data1 was the Microcontroller Pin number and data2 was the button state.

12:00:14.389 data1 =24 data2 = 0
12:00:14.389 data1 =24 data2 = 1
12:00:14.389 data1 =24 data2 = 0
12:00:15.851 data1 =24 data2 = 1
12:00:16.078 data1 =24 data2 = 0

We then extended the code to record the duration of each button press.

public class Program
{
   static DateTime buttonLastPressedAtUtc = DateTime.UtcNow;

   public static void Main()
   {
      InterruptPort button = new InterruptPort(Pins.ONBOARD_BTN, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);
      button.OnInterrupt += button_OnInterrupt;

      Thread.Sleep(Timeout.Infinite);
   }

   static void button_OnInterrupt(uint data1, uint data2, DateTime time)
   {
      if (data2 == 0)
      {
         TimeSpan duration = time - buttonLastPressedAtUtc;

         Debug.Print(duration.ToString());
      }
      else
      {
         buttonLastPressedAtUtc = time;
      }
   }
}

The thread ” (0x4) has exited with code 0 (0x0).
00:00:00.2031790
00:00:00.1954150
00:00:00.1962350

The next step was to keep track of the total duration of the button presses since the program started executing.

public class Program
{
   static DateTime buttonLastPressedAtUtc = DateTime.UtcNow;
   static TimeSpan buttonPressedDurationTotal;

   public static void Main()
   {
      InterruptPort button = new InterruptPort(Pins.ONBOARD_BTN, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);
      button.OnInterrupt += button_OnInterrupt;

      Thread.Sleep(Timeout.Infinite);
   }

   static void button_OnInterrupt(uint data1, uint data2, DateTime time)
   {
      if (data2 == 0)
      {
         TimeSpan duration = time - buttonLastPressedAtUtc;

         buttonPressedDurationTotal += duration;
          Debug.Print(duration.ToString() + " " + buttonPressedDurationTotal.ToString());
      }
      else
      {
         buttonLastPressedAtUtc = time;
      }
   }
}

The thread ” (0x4) has exited with code 0 (0x0).
00:00:00.2476460 00:00:00.2476460
00:00:00.2193600 00:00:00.4670060
00:00:00.2631400 00:00:00.7301460
00:00:00.0001870 00:00:00.7303330

We then added a timer to display the amount of time the button was pressed in the configured period.

public class Program
{
   static TimeSpan measurementDueTime = new TimeSpan(0, 0, 30);
   static TimeSpan measurementperiodTime = new TimeSpan(0, 0, 30);
   static DateTime buttonLastPressedAtUtc = DateTime.UtcNow;
   static TimeSpan buttonPressedDurationTotal;


   public static void Main()
   {
      InterruptPort button = new InterruptPort(Pins.GPIO_PIN_D5, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);
      button.OnInterrupt += button_OnInterrupt;

      Timer periodTimer = new Timer(periodTimerProc, button, measurementDueTime, measurementperiodTime);

      Thread.Sleep(Timeout.Infinite);
   }

   static void periodTimerProc(object status)
   {
      InterruptPort button = (InterruptPort)status;

      if (button.Read())
      {
         TimeSpan duration = DateTime.UtcNow - buttonLastPressedAtUtc;

         buttonPressedDurationTotal += duration; 
      }

      Debug.Print(buttonPressedDurationTotal.ToString());

      buttonPressedDurationTotal = new TimeSpan(0, 0, 0);
      buttonLastPressedAtUtc = DateTime.UtcNow;
   }

   static void button_OnInterrupt(uint data1, uint data2, DateTime time)
   {
      if (data2 == 0)
      {
         TimeSpan duration = time - buttonLastPressedAtUtc;

         buttonPressedDurationTotal += duration;

         Debug.Print(duration.ToString() + " " + buttonPressedDurationTotal.ToString());
      }
      else
      {
         buttonLastPressedAtUtc = time;
      }
   }
}

The thread ” (0x4) has exited with code 0 (0x0).
00:00:00
00:00:00
00:00:00.2299050 00:00:00.2299050
00:00:00.1956980 00:00:00.4256030
00:00:00.1693190 00:00:00.5949220
00:00:00.5949220

After some testing we identified that the handling of button presses at the period boundaries was problematic and revised the code some more. We added a timer for the startup period to simplify the interrupt handling code.

public class Program
{
   static TimeSpan measurementDueTime = new TimeSpan(0, 0, 60);
   static TimeSpan measurementperiodTime = new TimeSpan(0, 0, 30);
   static DateTime buttonLastPressedAtUtc = DateTime.UtcNow;
   static TimeSpan buttonPressedDurationTotal;

   public static void Main()
   {
      InterruptPort button = new InterruptPort(Pins.GPIO_PIN_D5, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);
      button.OnInterrupt += button_OnInterrupt;

      Timer periodTimer = new Timer(periodTimerProc, button, Timeout.Infinite, Timeout.Infinite);

      Timer startUpTImer = new Timer(startUpTimerProc, periodTimer, measurementDueTime.Milliseconds, Timeout.Infinite);

      Thread.Sleep(Timeout.Infinite);
   }

   static void startUpTimerProc(object status)
   {
      Timer periodTimer = (Timer)status;

      Debug.Print( DateTime.UtcNow.ToString("hh:mm:ss") + " -Startup complete");

      buttonLastPressedAtUtc = DateTime.UtcNow;
      periodTimer.Change(measurementDueTime, measurementperiodTime);
   }

   static void periodTimerProc(object status)
   {
      InterruptPort button = (InterruptPort)status;
      Debug.Print(DateTime.UtcNow.ToString("hh:mm:ss") + " -Period timer");

      if (button.Read())
      {
         TimeSpan duration = DateTime.UtcNow - buttonLastPressedAtUtc;

         buttonPressedDurationTotal += duration;
      }

      Debug.Print(buttonPressedDurationTotal.ToString());

      buttonPressedDurationTotal = new TimeSpan(0, 0, 0);
      buttonLastPressedAtUtc = DateTime.UtcNow;
   }

   static void button_OnInterrupt(uint data1, uint data2, DateTime time)
   {
      Debug.Print(DateTime.UtcNow.ToString("hh:mm:ss") + " -OnInterrupt");

      if (data2 == 0)
      {
         TimeSpan duration = time - buttonLastPressedAtUtc;

         buttonPressedDurationTotal += duration;

         Debug.Print(duration.ToString() + " " + buttonPressedDurationTotal.ToString());
      }
      else
      {
         buttonLastPressedAtUtc = time;
      }
   }
}

The debugging output looked positive, but more testing is required.

The thread ” (0x2) has exited with code 0 (0x0).
12:00:13 -Startup complete
12:01:13 -Period timer
00:00:00
12:01:43 -Period timer
00:00:00
12:01:46 -OnInterrupt
12:01:48 -OnInterrupt
00:00:01.2132510 00:00:01.2132510
12:01:49 -OnInterrupt
12:01:50 -OnInterrupt
00:00:01.3001240 00:00:02.5133750
12:01:53 -OnInterrupt
12:01:54 -OnInterrupt
00:00:01.1216510 00:00:03.6350260
12:02:13 -Period timer
00:00:03.6350260

Next steps – multi threading, extract code into a device driver and extend to support sensors like the SeeedStudio Smart dust Sensor which has two digital outputs, one for small particles (e.g. smoke) the other for larger particles (e.g. dust).

Netduino 3 Wifi pollution Sensor Part 1

I am working on a Netduino 3 Wifi based version for my original concept as a STEM project for high school students. I wanted to be able to upload data to a Microsoft Azure Eventhub or other HTTPS secured RESTful endpoint (e.g. xivelyIOT) to show how to build a securable solution. This meant a Netduino 3 Wifi device with the TI C3100 which does all the crypto processing was necessary.

The aim was to (over a number of blog posts) build a plug ‘n play box that initially was for measuring airborne particulates and then overtime add more sensors e.g. atmospheric gas concentrations, (Grove multichannel gas sensor), an accelerometer for earthquake early warning/monitoring (Grove 3-Axis Digital Accelerometer) etc.

Netduino 3 Wifi based pollution sensor

Bill of materials for prototype as at (October 2015)

  • Netduino 3 Wifi USD69.95
  • Seeedstudio Grove base shield V2 USD8.90
  • Seeedstudio Grove smart dust sensor USD16.95
  • Seeedstudio Grove Temperature & Humidity Sensor pro USD14.90
  • Seeedstudio ABS outdoor waterproof case USD1.65
  • Seeedstudio Grove 4 pin female to Grove 4 pin conversion cable USD3.90
  • Seeedstudio Grove 4 pin buckled 5CM cabed USD1.90

After the first assembly I have realised the box is a bit small. There is not a lot of clearance around the Netduino board (largely due to the go!bus connectors on the end making it a bit larger than a standard *duino board) and the space for additional sensors is limited so I will need to source a larger enclosure.

The dust sensor doesn’t come with a cable so I used the conversion cable instead. NOTE – The pins on the sensor are numbered right->Left rather than left->right.

The first step is to get the temperature and humidity sensor working with my driver code, then adapt the Seeedstudio Grove-Dust sensor code for the dual outputs of the SM-PWM-01 device.

According to the SM-PWM-01A device datasheet The P1 output is for small particles < 1uM (smoke) and P2 output is for large particles > 2uM (dust). The temperature & humidity sensor is included in the first iteration as other researchers have indicated that humidity levels can impact on the accuracy of optical particle counters.

Then, once the sensors are working as expected I will integrate a cut back version of the AMQPNetLite code and configuration storage code I wrote for my Netduino 3 wifi Azure EventHub Field Gateway.

Netduino 3 Wifi Azure Event Hub Field Gateway V2.0

After some testing I have improved the error handling and robustness of my Netduino 3 wifi based Azure Eventhub field gateway.

private void OnReceive(byte[] data)
{
   activityLed.Write(!activityLed.Read());

   // Ensure that we have a payload
   if (data.Length < 1 ) { Debug.Print( "ERROR - Message has no payload" ) ; return ; } string message = new String(Encoding.UTF8.GetChars(data)); Debug.Print(DateTime.UtcNow.ToString("HH:mm:ss") + " L=" + data.Length + " M=" + message); Thread thread = new Thread(() => EventHubSendMessage( data));
   thread.Start();
}

private void EventHubSendMessage( byte[] messageBody)
{
   #region Diagnostic assertions
   Debug.Assert(eventHubName != null);
   Debug.Assert(deviceId != null);
   Debug.Assert(gatewayId != null);
   Debug.Assert(messageBody != null);
   Debug.Assert(messageBody.Length > 0);
   #endregion

   if ((connection == null) || (session == null ) || (senderLink == null ))
   {
      lock (lockThis)
      {
         if (connection == null)
         {
            Debug.Print("AMQP Establish connection");
            try
            {
               connection = new Connection(new Address(serviceBusHost, serviceBusPort, serviceBusSasKeyName, serviceBusSasKey));

               connection.Closed = ConnectionClosedCallback;

               Debug.Print("AMQP Establish connection done");
            }
            catch (Exception ex)
            {
               Debug.Print("ERROR: AMQP Establish connection: " + ex.Message);
            }
         }

         if (connection == null)
         {
            return;
         }

         if (session == null)
         {
            Debug.Print("AMQP Establish session");
            try
            {
               session = new Session(connection);

               session.Closed = SessionClosedCallback;

               Debug.Print("AMQP Establish session done");
            }
            catch (Exception ex)
            {
               Debug.Print("ERROR: AMQP Establish session: " + ex.Message);
            }
         }

         if (session == null)
         {
            return;
         }

         if (senderLink == null)
         {
            Debug.Print("AMQP Establish SenderLink");
            try
            {
               senderLink = new SenderLink(session, "send-link", eventHubName);

               senderLink.Closed = SenderLinkClosedCallback;

               Debug.Print("AMQP Establish SenderLink done");
            }
            catch (Exception ex)
            {
               Debug.Print("ERROR: AMQP Establish SenderLink: " + ex.Message);
            }
         }

         if (senderLink == null)
         {
            return;
         }
      }
   }

         
   try
   {
      Debug.Print("AMQP Send start");
      DateTime startAtUtc = DateTime.UtcNow;

      Message message = new Message()
      {
         BodySection = new Data()
         {
            Binary = messageBody
         },
         ApplicationProperties = new Amqp.Framing.ApplicationProperties(),
      };

      message.ApplicationProperties["UploadedAtUtc"] = DateTime.UtcNow;
      message.ApplicationProperties["GatewayId"] = gatewayId;
      message.ApplicationProperties["DeviceId"] = deviceId;
      message.ApplicationProperties["EventId"] = Guid.NewGuid();

      senderLink.Send(message);
      DateTime finishAtUtc = DateTime.UtcNow;
      TimeSpan duration = finishAtUtc - startAtUtc;
      Debug.Print("AMQP Send done duration " + duration.ToString());
   }
   catch (Exception ex)
   {
      Debug.Print("ERROR: Publish failed with error: " + ex.Message);
   }
}

The software is quite reliable, when my internet connection fails it recovers gracefully and resumes uploading events when connectivity is restored.

The only issue is when the wireless access point is restarted, when the device reconnects it locks up and doesn’t recover. I have posted in the Netduino forums and logged at issue at the Github Netduino wifi repository.

I have been exploring rebooting the device in the NetworkChange_NetworkAvailabilityChanged handler when connectivity is restored.

Based on my logging the sending of events is pretty quick and the threads are interleaved

03:20:59 L=25 M={“D”:2,”H”:63.0,”T”:18.8}
AMQP Send start
03:20:59 L=25 M={“D”:1,”H”:54.5,”T”:18.7}
AMQP Send start
03:20:59 L=17 M={“D”:10,”P”:27.9}
AMQP Send start
AMQP Send done duration 00:00:00.2738220
AMQP Send done duration 00:00:00.4709960
AMQP Send done duration 00:00:01.0813910
03:21:01 L=17 M={“D”:10,”P”:27.4}
AMQP Send start
AMQP Send done duration 00:00:00.2820090
03:21:03 L=17 M={“D”:10,”P”:26.9}

Here is the code with usual caveats.

Next steps queuing messages in memory and then on the MicroSD card.