Netduino 3 Wifi Azure Event Hub Field Gateway V2.0

After some testing I have improved the error handling and robustness of my Netduino 3 wifi based Azure Eventhub field gateway.

private void OnReceive(byte[] data)
{
   activityLed.Write(!activityLed.Read());

   // Ensure that we have a payload
   if (data.Length < 1 ) { Debug.Print( "ERROR - Message has no payload" ) ; return ; } string message = new String(Encoding.UTF8.GetChars(data)); Debug.Print(DateTime.UtcNow.ToString("HH:mm:ss") + " L=" + data.Length + " M=" + message); Thread thread = new Thread(() => EventHubSendMessage( data));
   thread.Start();
}

private void EventHubSendMessage( byte[] messageBody)
{
   #region Diagnostic assertions
   Debug.Assert(eventHubName != null);
   Debug.Assert(deviceId != null);
   Debug.Assert(gatewayId != null);
   Debug.Assert(messageBody != null);
   Debug.Assert(messageBody.Length > 0);
   #endregion

   if ((connection == null) || (session == null ) || (senderLink == null ))
   {
      lock (lockThis)
      {
         if (connection == null)
         {
            Debug.Print("AMQP Establish connection");
            try
            {
               connection = new Connection(new Address(serviceBusHost, serviceBusPort, serviceBusSasKeyName, serviceBusSasKey));

               connection.Closed = ConnectionClosedCallback;

               Debug.Print("AMQP Establish connection done");
            }
            catch (Exception ex)
            {
               Debug.Print("ERROR: AMQP Establish connection: " + ex.Message);
            }
         }

         if (connection == null)
         {
            return;
         }

         if (session == null)
         {
            Debug.Print("AMQP Establish session");
            try
            {
               session = new Session(connection);

               session.Closed = SessionClosedCallback;

               Debug.Print("AMQP Establish session done");
            }
            catch (Exception ex)
            {
               Debug.Print("ERROR: AMQP Establish session: " + ex.Message);
            }
         }

         if (session == null)
         {
            return;
         }

         if (senderLink == null)
         {
            Debug.Print("AMQP Establish SenderLink");
            try
            {
               senderLink = new SenderLink(session, "send-link", eventHubName);

               senderLink.Closed = SenderLinkClosedCallback;

               Debug.Print("AMQP Establish SenderLink done");
            }
            catch (Exception ex)
            {
               Debug.Print("ERROR: AMQP Establish SenderLink: " + ex.Message);
            }
         }

         if (senderLink == null)
         {
            return;
         }
      }
   }

         
   try
   {
      Debug.Print("AMQP Send start");
      DateTime startAtUtc = DateTime.UtcNow;

      Message message = new Message()
      {
         BodySection = new Data()
         {
            Binary = messageBody
         },
         ApplicationProperties = new Amqp.Framing.ApplicationProperties(),
      };

      message.ApplicationProperties["UploadedAtUtc"] = DateTime.UtcNow;
      message.ApplicationProperties["GatewayId"] = gatewayId;
      message.ApplicationProperties["DeviceId"] = deviceId;
      message.ApplicationProperties["EventId"] = Guid.NewGuid();

      senderLink.Send(message);
      DateTime finishAtUtc = DateTime.UtcNow;
      TimeSpan duration = finishAtUtc - startAtUtc;
      Debug.Print("AMQP Send done duration " + duration.ToString());
   }
   catch (Exception ex)
   {
      Debug.Print("ERROR: Publish failed with error: " + ex.Message);
   }
}

The software is quite reliable, when my internet connection fails it recovers gracefully and resumes uploading events when connectivity is restored.

The only issue is when the wireless access point is restarted, when the device reconnects it locks up and doesn’t recover. I have posted in the Netduino forums and logged at issue at the Github Netduino wifi repository.

I have been exploring rebooting the device in the NetworkChange_NetworkAvailabilityChanged handler when connectivity is restored.

Based on my logging the sending of events is pretty quick and the threads are interleaved

03:20:59 L=25 M={“D”:2,”H”:63.0,”T”:18.8}
AMQP Send start
03:20:59 L=25 M={“D”:1,”H”:54.5,”T”:18.7}
AMQP Send start
03:20:59 L=17 M={“D”:10,”P”:27.9}
AMQP Send start
AMQP Send done duration 00:00:00.2738220
AMQP Send done duration 00:00:00.4709960
AMQP Send done duration 00:00:01.0813910
03:21:01 L=17 M={“D”:10,”P”:27.4}
AMQP Send start
AMQP Send done duration 00:00:00.2820090
03:21:03 L=17 M={“D”:10,”P”:26.9}

Here is the code with usual caveats.

Next steps queuing messages in memory and then on the MicroSD card.

Netduino 3 Wifi Azure Event Hub Field Gateway V1.0

The Netduino 3 Wifi device supports TLS connectivity and looked like it could provide a low power consumption field gateway to an Azure EventHub for my nRF24L01 equipped Netduino, Arduino & devDuino 1.3, 2.0 & 3.0 devices.

Netduino 3 Wifi Azure Event Hub Field Gateway

Netduino 3 Wifi Azure Field Gateway and a selection of arduino & devDuino devices

Bill of materials for field gateway prices as at (Sept 2015)

  • Netduino 3 Wifi USD69.95
  • SeeedStudio Solar Shield USD13.95
  • Lithium Ion 3000mAH battery USD15.00
  • Embedded coolness nRF24L01 shield with high power module USD17.85

The software uses AMQPNetLite which provides a lightweight implementation of the AMQP protocol (on the .Net framework, .Net Compact Framework, .Net Micro Framework, and WindowsPhone platforms) and the Nordic nRF24L01 Net Micro Framework Driver.The first version of the software is a proof of concept and over time I will add functionality and improve the reliability.

On application start up the nRF24L01, Azure Event Hub and network settings are loaded from the built in MicroSD card.

// Write empty template of the configuration settings to the SD card if pin D0 is high
if (!File.Exists(Path.Combine("\\sd", "app.config")))
{
   Debug.Print("Writing template configuration file then stopping");

   ConfigurationFileGenerate();

   Thread.Sleep(Timeout.Infinite);
}
appSettings.Load();

If there is no configuration file on the MicroSD card an empty template is created.

private void ConfigurationFileGenerate()
{
   // Write empty configuration file
   appSettings.SetString(nRF2L01AddressSetting, "Base1");
   appSettings.SetString(nRF2L01ChannelSetting, "10");
   appSettings.SetString(nRF2L01DataRateSetting, "0");

   appSettings.SetString(serviceBusHostSetting, "serviceBusHost");
   appSettings.SetString(serviceBusPortSetting, "5671");
   appSettings.SetString(serviceBusSasKeyNameSetting, "serviceBusSasKeyName");
   appSettings.SetString(serviceBusSasKeySetting, "serviceBusSasKey");
   appSettings.SetString(eventHubNameSetting, "eventHubName");

   appSettings.Save();
}

Once the Wifi connection has been established the device connects to a specified NTP server so any messages have an accurate timestamp and then initiates an AMQP connection.

Debug.Print("Network time");
try
{
   DateTime networkTime = NtpClient.GetNetworkTime(ntpServerHostname);
   Microsoft.SPOT.Hardware.Utility.SetLocalTime(networkTime);
   Debug.Print(networkTime.ToString(" dd-MM-yy HH:mm:ss"));
}
catch (Exception ex)
{
   Debug.Print("ERROR: NtpClient.GetNetworkTime: " + ex.Message);
   Thread.Sleep(Timeout.Infinite);
}
Debug.Print("Network time done");

// Connect to AMQP gateway
Debug.Print("AMQP Establish connection");
try
{
   Address address = new Address(serviceBusHost, serviceBusPort, serviceBusSasKeyName, serviceBusSasKey);
   connection = new Connection(address);
}
catch (Exception ex)
{
   Debug.Print("ERROR: AMQP Establish connection: " + ex.Message);
   Thread.Sleep(Timeout.Infinite);
}
Debug.Print("AMQP Establish connection done");

After the device has network connectivity, downloaded the correct time and connected to AMQP hub the nRF241L01 device is initialised.

The first version of the software starts a new thread to handle each message and handles connectivity failures badly. These issues and features like local queuing of messages will be added in future iterations.

private void OnReceive(byte[] data)
{
   activityLed.Write(!activityLed.Read());

   // Ensure that we have a payload
   if (data.Length < 1 ) { Debug.Print( "ERROR - Message has no payload" ) ; return ; } string message = new String(Encoding.UTF8.GetChars(data)); Debug.Print(DateTime.UtcNow.ToString("HH:mm:ss") + " " + gatewayId + " " + data.Length + " " + message); Thread thread = new Thread(() => EventHubSendMessage(connection, eventHubName, deviceId, gatewayId, data));
   thread.Start();
}



private void EventHubSendMessage(Connection connection, string eventHubName, string deviceId, string gatewayId, byte[] messageBody)
{
   try
   {
      Session session = new Session(connection);
      SenderLink sender = new SenderLink(session, "send-link", eventHubName);

      Message message = new Message()
      {
         BodySection = new Data()
         {
            Binary = messageBody
         },
         ApplicationProperties = new Amqp.Framing.ApplicationProperties(),
      };

      message.ApplicationProperties["UploadedAtUtc"] = DateTime.UtcNow;
      message.ApplicationProperties["GatewayId"] = gatewayId;
      message.ApplicationProperties["DeviceId"] = deviceId;
      message.ApplicationProperties["EventId"] = Guid.NewGuid().ToString();

      sender.Send(message);

      sender.Close();
      session.Close();
      }
   catch (Exception ex)
   {
      Debug.Print("ERROR: Publish failed with error: " + ex.Message);
   }
}

Initially the devices send events with a JSON payload.

ServiceBus Explorer

JSON Event messages displayed in ServiceBus Explorer

The code is available NetduinoNRF24L01AMQPNetLiteAzureEventHubGatewayV1.0 and when I have a spare afternoon I will upload to github.

MS Ignite Auckland NZ Presentation now available online

My presentation “All your device are belong to us” [M240] is now online at MSDN Channel 9

So much hype, so many different devices, so many protocols, so much data, so little security, welcome to the Internet of Things. Come and see how you can build an affordable, securable, scalable, interoperable, robust & reliable solution with embedded devices, Windows 10 IoT and Microsoft Azure. By 2020 there will be 26 Billion devices and 4.5 million developers building solutions so the scope is limitless.

I had 8 devices in my presentation so the scope for disaster was high.

The first demo was of how sensors could be connected across Arduino, Netduino and Raspberry PI platforms.

The Arduino demo used

The Netduino demo used

The Raspbery PI Windows 10 IoT Core demo used

The hobbyist data acquisition demo collected data from two devduino devices that were in passed around by the audience and were each equipped with a Temperature & Humidity sensor. They uploaded data to Xively over an NRF24L01 link to a gateway running on a Netduino 3 Ethernet and the data was displayed in real-time on my house information page

The professional data acquisition demo uploaded telemetry data to an Azure ServiceBus EventHub and retrieved commands from an Azure ServiceBus Queue. Both devices were running software based on Azure ServiceBus Lite by Paolo Paiterno

The telemetry stream was the temperature of some iced water.

The commands were processed by a Raspbery PI running Windows 10 IoT Core which turned a small fan on & off to illustrate how a FrostFan could be used in a vineyard to reduce frost damage to the vines.

Frost Fan demo

MS Ignite 2015 Frost Fan demo

My demos all worked on the day which was a major win as many other presenters struggled with connectivity. Thanks to the conference infrastructure support guys who helped me sort things out.

With the benefit of hindsight, I tried to fit too much in and the overnight partial rewrite post attending the presentation Mashup the Internet of Things, Azure App Service and Windows 10 to Deliver Business Value [M387] by Rob Tiffany was a bit rushed.