Wireless field gateway protocol V2

I have now built a couple of nRF2L01P field gateways (for AdaFriut.IO & Azure IoT Hubs) which run as a background tasks on Windows 10 IoT Core on RaspberyPI). I have also written several clients which run on Arduino, devDuino, Netduino, and Seeeduino devices.

I have tried to keep the protocol simple (telemetry only) to deploy and it will be used in high school student projects in the next couple of weeks.

To make the payload smaller the first byte of the message now specifies the message type in the top nibble and the length of the device unique identifier in the bottom nibble.

0 = Echo

The message is displayed by the field gateway as text & hexadecimal.

1 = Device identifier + Comma separated values (CSV) payload

[0] – Set to 0001, XXXX   Device identifier length

[1]..[1+Device identifier length] – Unique device identifier bytes e.g. Mac address

[1+Device identifier length+1 ]..[31] – CSV payload e.g.  SensorID value, SensorID value

 

Wireless field gateway Netduino client V1

This client is a Netduino V2Plus/V3 Ethernet/V3 Wifi device with a Silicon Labs SI7005 temperature & humidity sensor. These devices when used as sensor nodes can be battery powered and I use the Mac Address as the unique device identifier.

Reducing the power consumption, improving reliability etc. will be covered in future posts

//---------------------------------------------------------------------------------
// Copyright (c) 2017, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using System.Net;
using System.Text;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using Microsoft.SPOT.Net.NetworkInformation;
using devMobile.NetMF.Sensor;
using Gralin.NETMF.Nordic;
using SecretLabs.NETMF.Hardware.Netduino;

namespace devMobile.IoT.FIeldGateway.Netduino.Client
{
   class Client
   {
      private const byte nRF24Channel = 10;
      private const NRFDataRate nRF24DataRate = NRFDataRate.DR250kbps;
      private readonly byte[] nRF24ClientAddress = Encoding.UTF8.GetBytes("TandH");
      private readonly byte[] nRF24BaseStationAddress = Encoding.UTF8.GetBytes("Base1");
      private static byte[] deviceIdentifier;
      private readonly OutputPort led = new OutputPort(Pins.ONBOARD_LED, false);
      private readonly NRF24L01Plus radio;
      private readonly SiliconLabsSI7005 sensor = new SiliconLabsSI7005();

      public Client()
      {
         radio = new NRF24L01Plus();
      }

      public void Run()
      {
         // Configure the nRF24 hardware
         radio.OnDataReceived += OnReceive;
         radio.OnTransmitFailed += OnSendFailure;
         radio.OnTransmitSuccess += OnSendSuccess;

         radio.Initialize(SPI.SPI_module.SPI1, Pins.GPIO_PIN_D7, Pins.GPIO_PIN_D3, Pins.GPIO_PIN_D2);
         radio.Configure(nRF24ClientAddress, nRF24Channel, nRF24DataRate);
         radio.Enable();

         // Setup the device unique identifer, in this case the hardware MacAddress
         deviceIdentifier = NetworkInterface.GetAllNetworkInterfaces()[0].PhysicalAddress;
         Debug.Print(" Device Identifier : " + BytesToHexString(deviceIdentifier));

         Timer humidityAndtemperatureUpdates = new Timer(HumidityAndTemperatureTimerProc, null, 15000, 15000);

         Thread.Sleep(Timeout.Infinite);
      }

      private void HumidityAndTemperatureTimerProc(object state)
      {
         led.Write(true);

         double humidity = sensor.Humidity();
         double temperature = sensor.Temperature();

         Debug.Print("H:" + humidity.ToString("F1") + " T:" + temperature.ToString("F1"));
         string values = "T " + temperature.ToString("F1") + ",H " + humidity.ToString("F0");

         // Stuff the 2 byte header ( payload type & deviceIdentifierLength ) + deviceIdentifier into payload
         byte[] payload = new byte[1 + 1 + deviceIdentifier.Length + values.Length];
         payload[0] = 1;
         payload[1] = (byte)deviceIdentifier.Length;
         Array.Copy(deviceIdentifier, 0, payload, 2, deviceIdentifier.Length);

         Encoding.UTF8.GetBytes( values, 0, values.Length, payload, 8 ) ;

         radio.SendTo(nRF24BaseStationAddress, payload );
      }

      private void OnSendSuccess()
      {
         led.Write(false);

         Debug.Print("Send Success!");
      }

      private void OnSendFailure()
      {
         Debug.Print("Send failed!");
      }

      private void OnReceive(byte[] data)
      {
         led.Write(!led.Read());

         string message = new String(Encoding.UTF8.GetChars(data));

         Debug.Print("Receive " + message); ;
      }

      private static string BytesToHexString(byte[] bytes)
      {
         string hexString = string.Empty;

         // Create a character array for hexidecimal conversion.
         const string hexChars = "0123456789ABCDEF";

         // Loop through the bytes.
         for (byte b = 0; b < bytes.Length; b++)          {             if (b > 0)
               hexString += "-";

            // Grab the top 4 bits and append the hex equivalent to the return string.
            hexString += hexChars[bytes[b] >> 4];

            // Mask off the upper 4 bits to get the rest of it.
            hexString += hexChars[bytes[b] & 0x0F];
         }

         return hexString;
      }
   }

.Net Micro framework Deployment Tool output

WindowsIoTCentralNetduinoClient

Raspberry PI UWP application output

Interrupt Triggered: FallingEdge
11:40:46 Address 5C-86-4A-00-E4-1D Length 6 Payload T 25.2,H 90 Length 11
 Sensor 5C-86-4A-00-E4-1D-T Value 25.2
 Sensor 5C-86-4A-00-E4-1D-H Value 90
Interrupt Triggered: RisingEdge
Interrupt Triggered: FallingEdge
11:41:01 Address 5C-86-4A-00-E4-1D Length 6 Payload T 25.3,H 91 Length 11
 Sensor 5C-86-4A-00-E4-1D-T Value 25.3
 Sensor 5C-86-4A-00-E4-1D-H Value 91
Interrupt Triggered: RisingEdge
Interrupt Triggered: FallingEdge
11:41:16 Address 5C-86-4A-00-E4-1D Length 6 Payload T 25.3,H 90 Length 11
 Sensor 5C-86-4A-00-E4-1D-T Value 25.3
 Sensor 5C-86-4A-00-E4-1D-H Value 90
Interrupt Triggered: RisingEdge
Interrupt Triggered: FallingEdge
11:41:31 Address 5C-86-4A-00-E4-1D Length 6 Payload T 25.3,H 90 Length 11
 Sensor 5C-86-4A-00-E4-1D-T Value 25.3
 Sensor 5C-86-4A-00-E4-1D-H Value 90
Interrupt Triggered: RisingEdge
Interrupt Triggered: FallingEdge
11:41:46 Address 5C-86-4A-00-E4-1D Length 6 Payload T 25.3,H 90 Length 11
 Sensor 5C-86-4A-00-E4-1D-T Value 25.3
 Sensor 5C-86-4A-00-E4-1D-H Value 90
Interrupt Triggered: RisingEdge

Bill of materials (prices as at Jan 2018)

Wireless field gateway protocol V1

I’m going to build a number of nRF2L01P field gateways (Netduino Ethernet & Wifi running .NetMF, Raspberry PI running Windows 10 IoT Core, RedBearLab 3200  etc.), clients which run on a variety of hardware (Arduino, devDuino, Netduino, Seeeduino etc.) which, then upload data to a selection of IoT Cloud services (AdaFruit.IO, ThingSpeak, Microsoft IoT Central etc.)

The nRF24L01P is widely supported with messages up to 32 bytes long, low power consumption and 250kbps, 1Mbps and 2Mbps data rates.

The aim is to keep the protocol simple (telemetry only initially) to implement and debug as the client side code will be utilised by high school student projects.

The first byte of the message specifies the message type

0 = Echo

The message is displayed by the field gateway as text & hexadecimal.

1 = Device identifier + Comma separated values (CSV) payload

[0] – Set to 1

[1] – Device identifier length

[2]..[2+Device identifier length] – Unique device identifier bytes e.g. Mac address

[2+Device identifier length+1 ]..[31] – CSV payload e.g.  SensorID value, SensorID value

Overtime I will support more message types and wireless protocols.

 

AdaFruit IO basic Netduino HTTP client

I use Netduino devices for teaching and my students often build projects which need a cloud based service like AdaFruit.IO to capture, store and display their sensor data.

My Proof of Concept (PoC) which uses a slightly modified version of the AdaFruit.IO basic desktop HTTP client code has been running on several Netduino 2 Plus, Netduino 3 Ethernet and Netduino 3 Wifi devices for the last couple of days and looks pretty robust.

The Netduino 3 Wifi device also supports https for improved security and privacy. They also make great field gateways as they can run off solar/battery power.

N2PN3WDashBoard

The devices have been uploading temperature and humidity measurements from a Silicon labs Si7005 sensor. (Outside sensor suffering from sunstrike)

N3WifiTemperatureAndHumiditySensor

program.cs

*

Copyright ® 2017 December devMobile Software, All Rights Reserved

THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR
PURPOSE.

http://www.devmobile.co.nz

*/
using System;
using System.Net;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using Microsoft.SPOT.Net.NetworkInformation;
using SecretLabs.NETMF.Hardware.Netduino;
using devMobile.NetMF.Sensor;
using devMobile.IoT.NetMF;

namespace devMobile.IoT.AdaFruitIO.NetMF.Client
{
   public class Program
   {
      private const string adaFruitIOApiBaseUrl = @"https://IO.adafruit.com/api/v2/";
      private const string group = "netduino3";
      private const string temperatureFeedKey = "t";
      private const string humidityFeedKey = "h";
      private const string adaFruitUserName = "YourUserName";
      private const string adaFruitIOApiKey = "YourAPIKey";
      private static readonly TimeSpan timerDueAfter = new TimeSpan(0, 0, 15);
      private static readonly TimeSpan timerPeriod = new TimeSpan(0, 0, 30);
      private static OutputPort led = new OutputPort(Pins.ONBOARD_LED, false);
      private static SiliconLabsSI7005 sensor = new SiliconLabsSI7005();
      private static AdaFruitIoClient adaFruitIoClient = new AdaFruitIoClient(adaFruitUserName, adaFruitIOApiKey, adaFruitIOApiBaseUrl);

      public static void Main()
      {
         // Wait for Network address if DHCP
         NetworkInterface networkInterface = NetworkInterface.GetAllNetworkInterfaces()[0];
         if (networkInterface.IsDhcpEnabled)
         {
            Debug.Print(" Waiting for DHCP IP address");

            while (NetworkInterface.GetAllNetworkInterfaces()[0].IPAddress == IPAddress.Any.ToString())
            {
               Debug.Print(" .");
               led.Write(!led.Read());
               Thread.Sleep(250);
            }
            led.Write(false);
         }

         // Display network config for debugging
         Debug.Print("Network configuration");
         Debug.Print(" Network interface type : " + networkInterface.NetworkInterfaceType.ToString());
         Debug.Print(" MAC Address : " + BytesToHexString(networkInterface.PhysicalAddress));
         Debug.Print(" DHCP enabled : " + networkInterface.IsDhcpEnabled.ToString());
         Debug.Print(" Dynamic DNS enabled : " + networkInterface.IsDynamicDnsEnabled.ToString());
         Debug.Print(" IP Address : " + networkInterface.IPAddress.ToString());
         Debug.Print(" Subnet Mask : " + networkInterface.SubnetMask.ToString());
         Debug.Print(" Gateway : " + networkInterface.GatewayAddress.ToString());

         foreach (string dnsAddress in networkInterface.DnsAddresses)
         {
            Debug.Print(" DNS Server : " + dnsAddress.ToString());
         }

         Timer humidityAndtemperatureUpdates = new Timer(HumidityAndTemperatureTimerProc, null, timerDueAfter, timerPeriod);

         Thread.Sleep(Timeout.Infinite);
      }

      static private void HumidityAndTemperatureTimerProc(object state)
      {
         led.Write(true);

         try
         {
            double humidity = sensor.Humidity();

            Debug.Print(" Humidity " + humidity.ToString("F0") + "%");
            adaFruitIoClient.FeedUpdate(group, humidityFeedKey, humidity.ToString("F0"));
         }
         catch (Exception ex)
         {
            Debug.Print("Humidifty read+update failed " + ex.Message);

            return;
         }

         try
         {
            double temperature = sensor.Temperature();

            Debug.Print(" Temperature " + temperature.ToString("F1") + "°C");
            adaFruitIoClient.FeedUpdate(group, temperatureFeedKey, temperature.ToString("F1"));
         }
         catch (Exception ex)
         {
            Debug.Print("Temperature read+update failed " + ex.Message);

            return;
         }

         led.Write(false);
      }

      private static string BytesToHexString(byte[] bytes)
      {
         string hexString = string.Empty;

         // Create a character array for hexidecimal conversion.
         const string hexChars = "0123456789ABCDEF";

         // Loop through the bytes.
         for (byte b = 0; b < bytes.Length; b++)          {             if (b > 0)
               hexString += "-";

            // Grab the top 4 bits and append the hex equivalent to the return string.
            hexString += hexChars[bytes[b] >> 4];

            // Mask off the upper 4 bits to get the rest of it.
            hexString += hexChars[bytes[b] & 0x0F];
         }

         return hexString;
      }
   }
}

AdaFruit.IO client.cs, handles feed groups and individual feeds

/*

Copyright ® 2017 December devMobile Software, All Rights Reserved

THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR
PURPOSE.

http://www.devmobile.co.nz

*/
using System;
using System.IO;
using System.Net;
using System.Text;
using Microsoft.SPOT;

namespace devMobile.IoT.NetMF
{
   public class AdaFruitIoClient
   {
      private const string apiBaseUrlDefault = @"http://IO.adafruit.com/api/v2/";
      private string apiBaseUrl = "";
      private string userName = "";
      private string apiKey = "";
      private int httpRequestTimeoutmSec;
      private int httpRequestReadWriteTimeoutmSec;

      public AdaFruitIoClient(string userName, string apiKey, string apiBaseUrl = apiBaseUrlDefault, int httpRequestTimeoutmSec = 2500, int httpRequestReadWriteTimeoutmSec = 5000)
      {
         this.apiBaseUrl = apiBaseUrl;
         this.userName = userName;
         this.apiKey = apiKey;
         this.httpRequestReadWriteTimeoutmSec = httpRequestReadWriteTimeoutmSec;
         this.httpRequestTimeoutmSec = httpRequestTimeoutmSec;
      }

      public void FeedUpdate(string group, string feedKey, string value)
      {
         string feedUrl;

         if (group.Trim() == string.Empty)
         {
            feedUrl = apiBaseUrl + userName + @"/feeds/" + feedKey + @"/data";
         }
         else
         {
            feedUrl = apiBaseUrl + userName + @"/feeds/" + group.Trim() + "." + feedKey + @"/data";
         }

         HttpWebRequest request = (HttpWebRequest)WebRequest.Create(feedUrl);
         {
            string payload = @"{""value"": """ + value + @"""}";
            byte[] buffer = Encoding.UTF8.GetBytes(payload);

            DateTime httpRequestedStartedAtUtc = DateTime.UtcNow;

            request.Method = "POST";
            request.ContentLength = buffer.Length;
            request.ContentType = @"application/json";
            request.Headers.Add("X-AIO-Key", apiKey);
            request.KeepAlive = false;
            request.Timeout = this.httpRequestTimeoutmSec;
            request.ReadWriteTimeout = this.httpRequestReadWriteTimeoutmSec;

            using (Stream stream = request.GetRequestStream())
            {
               stream.Write(buffer, 0, buffer.Length);
            }

            using (var response = (HttpWebResponse)request.GetResponse())
            {
               Debug.Print(" Status: " + response.StatusCode + " : " + response.StatusDescription);
            }

            TimeSpan duration = DateTime.UtcNow - httpRequestedStartedAtUtc;
            Debug.Print(" Duration: " + duration.ToString());
         }
      }
   }
}

Bill of materials for PoC

nRF24 Windows 10 IoT Core reboot

My first live deployment of the nRF24L01 Windows 10 IoT Core field gateway is now scheduled for mid Q1 2018 so time for a reboot. After digging out my Raspbery PI 2/3 devices and the nRF24L01+ shield (with modifications detailed here) I have a basic plan with some milestones.

My aim is to be able to wirelessly acquire data from several dozen Arduino, devduino, seeeduino, and Netduino devices, Then, using a field gateway on a Raspberry PI running Windows 10 IoT Core upload it to Microsoft IoT Central

First bit of code – Bleepy a simple background application to test the piezo beeper on the RPI NRF24 Shield

namespace devmobile.IoTCore.Bleepy
{
   public sealed class StartupTask : IBackgroundTask
   {
      private BackgroundTaskDeferral deferral;
      private const int ledPinNumber = 4;
      private GpioPin ledGpioPin;
      private ThreadPoolTimer timer;

      public void Run(IBackgroundTaskInstance taskInstance)
      {
         var gpioController = GpioController.GetDefault();
         if (gpioController == null)
         {
            Debug.WriteLine("GpioController.GetDefault failed");
            return;
         }

         ledGpioPin = gpioController.OpenPin(ledPinNumber);
         if (ledGpioPin == null)
         {
            Debug.WriteLine("gpioController.OpenPin failed");
            return;
         }

         ledGpioPin.SetDriveMode(GpioPinDriveMode.Output);

         this.timer = ThreadPoolTimer.CreatePeriodicTimer(Timer_Tick, TimeSpan.FromMilliseconds(500));

         deferral = taskInstance.GetDeferral();

         Debug.WriteLine("Rum completed");
      }

      private void Timer_Tick(ThreadPoolTimer timer)
      {
         GpioPinValue currentPinValue = ledGpioPin.Read();

         if (currentPinValue == GpioPinValue.High)
         {
            ledGpioPin.Write(GpioPinValue.Low);
         }
         else
         {
            ledGpioPin.Write(GpioPinValue.High);
         }
      }
   }
}

Note the blob of blu tack over the piezo beeper to mute noise
nRF24ShieldMuted

nRF24 Windows 10 IoT Core Test Harness

After modifying the Raspbery PI nRF24L01 shields I built a single page single button Universal Windows Platforms(UWP) test harness (using the techfooninja RF24 library) to check everything was working as expected.

I used a couple of Netduinos and Raspbery PI devices to as test clients.

public sealed partial class MainPage : Page
{
   private const byte ChipEnablePin = 25;
   private const byte ChipSelectPin = 0;
   private const byte InterruptPin = 17;
   private const byte Channel = 10;
   private RF24 radio;

   public MainPage()
   {
      this.InitializeComponent();

      this.radio = new RF24();

      this.radio.OnDataReceived += this.Radio_OnDataReceived;
      this.radio.OnTransmitFailed += this.Radio_OnTransmitFailed;
      this.radio.OnTransmitSuccess += this.Radio_OnTransmitSuccess;

      this.radio.Initialize(ChipEnablePin, ChipSelectPin, InterruptPin);
      this.radio.Address = Encoding.UTF8.GetBytes("Base1");
      this.radio.Channel = Channel;
      this.radio.PowerLevel = PowerLevel.Low;
      this.radio.DataRate = DataRate.DR250Kbps;

      this.radio.IsEnabled = true;

      Debug.WriteLine("Address: " + Encoding.UTF8.GetString(this.radio.Address));
      Debug.WriteLine("Channel: " + this.radio.Channel);
      Debug.WriteLine("DataRate: " + this.radio.DataRate);
      Debug.WriteLine("PA: " + this.radio.PowerLevel);
      Debug.WriteLine("IsAutoAcknowledge: " + this.radio.IsAutoAcknowledge);
      Debug.WriteLine("IsDynamicAcknowledge: " + this.radio.IsDynamicAcknowledge);
      Debug.WriteLine("IsDynamicPayload: " + this.radio.IsDynamicPayload);
      Debug.WriteLine("IsEnabled: " + this.radio.IsEnabled);
      Debug.WriteLine("IsInitialized: " + this.radio.IsInitialized);
      Debug.WriteLine("IsPowered: " + this.radio.IsPowered);
   }

   private void Radio_OnDataReceived(byte[] data)
   {
     string dataUTF8 = Encoding.UTF8.GetString(data);

     Debug.WriteLine(string.Format("Received: {0}", dataUTF8));
   }

   private void buttonSend_Click(object sender, Windows.UI.Xaml.RoutedEventArgs e)
   {
      this.radio.SendTo(Encoding.UTF8.GetBytes("Duino"), Encoding.UTF8.GetBytes(DateTime.UtcNow.ToString("yy-MM-dd hh:mm:ss"))) ;
   }

   private void Radio_OnTransmitSuccess()
   {
      Debug.WriteLine("Radio_OnTransmitSuccess");
   }

   private void Radio_OnTransmitFailed()
   {
      Debug.WriteLine("Radio_OnTransmitFailed");
   }
}

Interrupt Triggered: FallingEdge
Data Sent!
Radio_OnTransmitSuccess
Interrupt Triggered: RisingEdge
Interrupt Triggered: FallingEdge
Received: 20.4 70.7
Interrupt Triggered: RisingEdge
Interrupt Triggered: FallingEdge
Data Sent!
Radio_OnTransmitSuccess
Interrupt Triggered: RisingEdge
Interrupt Triggered: FallingEdge
Received: 20.3 70.8
Interrupt Triggered: RisingEdge

The Raspberry PI could reliably receive and transmit messages.

NetMF MP3 Player Part 2

Building on the file listing code from the previous post in the next class we wrote the simplest possible NetMF code to play all of MP3 files on an SD Card. The Mp3 player shields I have all use a Vs1053B chip to decode the MP3 byte stream. This sample uses the driver code from softelectrotech.

public static void Main()
{
   Vs1053B player = new Vs1053B(Pins.GPIO_PIN_D2, Pins.GPIO_PIN_D6, Pins.GPIO_PIN_D7, Pins.GPIO_PIN_D8);
   string[] MusicFiles = Directory.GetFiles(@"\SD");
   player.SetVolume(180, 180);

   // Print a list of all the files on the SD card.
   foreach (string file in MusicFiles)
   {
      Debug.Print(file);
   }

   foreach (string file in MusicFiles)
   {
      Debug.Print("Play start " + file);

      player.Play(file, true);

      Debug.Print("Play finish " + file);
   }
}

Some of the students observed the order of the files was not what they were expecting. The order of the files appeared to depend on how they were copied to the memory card.

\SD\05 Sunday Bloody Sunday.mp3
\SD\06 Bad.mp3
\SD\07 Where the Streets Have No Name.mp3
\SD\08 I Will Follow.mp3
\SD\09 The Unforgettable Fire.mp3
\SD\10 Sweetest Thing [The Single Mix].mp3
\SD\11 Desire.mp3
\SD\12 When Love Comes to Town.mp3
\SD\13 Angel of Harlem.mp3
\SD\14 All I Want Is You.mp3
\SD\01 Pride (In the Name of Love).mp3
\SD\02 New Year's Day.mp3
\SD\03 With or Without You.mp3
\SD\04 I Still Haven't Found What I'm Looking For.mp3
Playback
Play start \SD\05 Sunday Bloody Sunday.mp3
Play finish \SD\05 Sunday Bloody Sunday.mp3
Play start \SD\06 Bad.mp3

All of the students observed that the tacks were being played synchronously which didn’t allow you to change the volume or interrupt the playback to pause the current track or change the track being played.

The next steps were to ensure the music files were sorted into a consistent order and that playback was not synchronous.

MusicPlayerSyncBasic code