Adafruit.IO MQTT LoRa Field Gateway

Back in April I started working on an MQTT LoRa Field gateway which was going to support a selection of different Software as a service(SaaS) Internet of Things(IoT) platforms.

After a long pause in development I have a working AdaFruit.IO client and have 3 proof of concept (PoC) integrations for AskSensors, Losant and Ubidots. I am also working on Azure IoT Hub, Azure IoT Central and MyDevice Cayenne clients. The first iteration is focused on Device to Cloud (D2C) messaging in the next iteration I will add Cloud to Device where viable(C2D).

My applications use a lightweight, easy to implemented protocol which is intended for hobbyist and educational use rather than commercial applications (I have been working on a more secure version as yet another side project)

I have a number of sample Arduino with Dragino LoRa Shield for Arduino, MakerFabs Maduino, Dragino LoRa Mini Dev, M2M Low power Node and Netduino with Elecrow LoRa RFM95 Shield etc. clients. These work with both my platform specific (Adafruit.IO, Azure IoT Central) gateways and protocol specific field gateways.

Maduino client dashboard

When the application is first started it creates a minimal configuration file which should be downloaded, the missing information filled out, then uploaded using the File explorer in the Windows device portal.

{
  "MQTTUserName": "AdaFruitIO user",
  "MQTTPassword": "AIO Key",
  "MQTTClientID": "MQTTLoRaGateway",
  "MQTTServer": "io.adafruit.com",
  "Address": "LoRaIoT1",
  "Frequency": 915000000.0,
  "MessageHandlerAssembly": "Mqtt.IoTCore.FieldGateway.LoRa.Adafruit",
  "PlatformSpecificConfiguration": "mqttloragateway"
}

The application logs debugging information to the Windows 10 IoT Core ETW logging Microsoft-Windows-Diagnostics-LoggingChannel

MQTT LoRa Gateway with Adafruit.IO plug-in

The SaaS platform specific interface has gained an additional parameter for platform specific configuration.

namespace devMobile.Mqtt.IoTCore.FieldGateway
{
	using System;
	using Windows.Foundation.Diagnostics;

	using devMobile.IoT.Rfm9x;
	using MQTTnet;
	using MQTTnet.Client;

	public interface IMessageHandler
	{
		void Initialise(LoggingChannel logging, IMqttClient mqttClient, Rfm9XDevice rfm9XDevice,string platformSpecificConfiguration);

		void Rfm9XOnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e);

		void MqttApplicationMessageReceived(object sender, MqttApplicationMessageReceivedEventArgs e);

		void Rfm9xOnTransmit(object sender, Rfm9XDevice.OnDataTransmitedEventArgs e);
	}
}

This is used for the AdaFruit.IO GroupName so Adafruit.IO feed values are not all in a single group.

public class MessageHandler : IMessageHandler
	{
		private LoggingChannel Logging { get; set; }
		private IMqttClient MqttClient { get; set; }
		private Rfm9XDevice Rfm9XDevice { get; set; }
      private string PlatformSpecificConfiguration { get; set; }


      void IMessageHandler.Initialise(LoggingChannel logging, IMqttClient mqttClient, Rfm9XDevice rfm9XDevice, string platformSpecificConfiguration)
		{
			LoggingFields processInitialiseLoggingFields = new LoggingFields();

			this.Logging = logging;
			this.MqttClient = mqttClient;
			this.Rfm9XDevice = rfm9XDevice;
			this.PlatformSpecificConfiguration = platformSpecificConfiguration;
		}

		async void IMessageHandler.Rfm9XOnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e)
		{
			LoggingFields processReceiveLoggingFields = new LoggingFields();

			processReceiveLoggingFields.AddString("PacketSNR", e.PacketSnr.ToString("F1"));
			processReceiveLoggingFields.AddInt32("PacketRSSI", e.PacketRssi);
			processReceiveLoggingFields.AddInt32("RSSI", e.Rssi);

			string addressBcdText = BitConverter.ToString(e.Address);
			processReceiveLoggingFields.AddInt32("DeviceAddressLength", e.Address.Length);
			processReceiveLoggingFields.AddString("DeviceAddressBCD", addressBcdText);

			string payloadBcdText = BitConverter.ToString(e.Data);
			processReceiveLoggingFields.AddInt32("PayloadLength", e.Data.Length);
			processReceiveLoggingFields.AddString("DeviceAddressBCD", payloadBcdText);

			this.Logging.LogEvent("Rfm9XOnReceive", processReceiveLoggingFields, LoggingLevel.Information);
		}

		void IMessageHandler.MqttApplicationMessageReceived(object sender, MqttApplicationMessageReceivedEventArgs e)
		{
			LoggingFields processReceiveLoggingFields = new LoggingFields();

			processReceiveLoggingFields.AddString("ClientId", e.ClientId);
#if DEBUG
			processReceiveLoggingFields.AddString("Payload", e.ApplicationMessage.ConvertPayloadToString());
#endif
			processReceiveLoggingFields.AddString("QualityOfServiceLevel", e.ApplicationMessage.QualityOfServiceLevel.ToString());
			processReceiveLoggingFields.AddBoolean("Retain", e.ApplicationMessage.Retain);
			processReceiveLoggingFields.AddString("Topic", e.ApplicationMessage.Topic);

			this.Logging.LogEvent("MqttApplicationMessageReceived topic not processed", processReceiveLoggingFields, LoggingLevel.Error);
		}

		void IMessageHandler.Rfm9xOnTransmit(object sender, Rfm9XDevice.OnDataTransmitedEventArgs e)
		{
			this.Logging.LogMessage("Rfm9xOnTransmit", LoggingLevel.Information);
		}
	}
Adafruit.IO Group for a single field gateway
Group Setup

The message handler uploads all values in an inbound messages in one MQTT message using the AdaFruit.IO Group Feed format.

      async void IMessageHandler.Rfm9XOnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e)
      {
         LoggingFields processReceiveLoggingFields = new LoggingFields();
         char[] sensorReadingSeparators = { ',' };
         char[] sensorIdAndValueSeparators = { ' ' };

         processReceiveLoggingFields.AddString("PacketSNR", e.PacketSnr.ToString("F1"));
         processReceiveLoggingFields.AddInt32("PacketRSSI", e.PacketRssi);
         processReceiveLoggingFields.AddInt32("RSSI", e.Rssi);

         string addressBcdText = BitConverter.ToString(e.Address);
         processReceiveLoggingFields.AddInt32("DeviceAddressLength", e.Address.Length);
         processReceiveLoggingFields.AddString("DeviceAddressBCD", addressBcdText);

         string messageText;
         try
         {
            messageText = UTF8Encoding.UTF8.GetString(e.Data);
            processReceiveLoggingFields.AddString("MessageText", messageText);
         }
         catch (Exception ex)
         {
            processReceiveLoggingFields.AddString("Exception", ex.ToString());
            this.Logging.LogEvent("PayloadProcess failure converting payload to text", processReceiveLoggingFields, LoggingLevel.Warning);
            return;
         }

         // Chop up the CSV text
         string[] sensorReadings = messageText.Split(sensorReadingSeparators, StringSplitOptions.RemoveEmptyEntries);
         if (sensorReadings.Length < 1)
         {
            this.Logging.LogEvent("PayloadProcess payload contains no sensor readings", processReceiveLoggingFields, LoggingLevel.Warning);
            return;
         }

         JObject payloadJObject = new JObject();

         JObject feeds = new JObject();

         // Chop up each sensor read into an ID & value
         foreach (string sensorReading in sensorReadings)
         {
            string[] sensorIdAndValue = sensorReading.Split(sensorIdAndValueSeparators, StringSplitOptions.RemoveEmptyEntries);

            // Check that there is an id & value
            if (sensorIdAndValue.Length != 2)
            {
               this.Logging.LogEvent("PayloadProcess payload invalid format", processReceiveLoggingFields, LoggingLevel.Warning);
               return;
            }

            string sensorId = string.Concat(addressBcdText, sensorIdAndValue[0]);
            string value = sensorIdAndValue[1];

            feeds.Add(sensorId.ToLower(), value);
         }
         payloadJObject.Add("feeds", feeds);

         string topic = $"{MqttClient.Options.Credentials.Username}/groups/{PlatformSpecificConfiguration}";

         try
         {
            var message = new MqttApplicationMessageBuilder()
               .WithTopic(topic)
               .WithPayload(JsonConvert.SerializeObject(payloadJObject))
               .WithAtLeastOnceQoS()
               .Build();
            Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync start", DateTime.UtcNow);
            await MqttClient.PublishAsync(message);
            Debug.WriteLine(" {0:HH:mm:ss} MQTT Client PublishAsync finish", DateTime.UtcNow);

            this.Logging.LogEvent("PublishAsync Adafruit payload", processReceiveLoggingFields, LoggingLevel.Information);
         }
         catch (Exception ex)
         {
            processReceiveLoggingFields.AddString("Exception", ex.ToString());
            this.Logging.LogEvent("PublishAsync Adafruit payload", processReceiveLoggingFields, LoggingLevel.Error);
         }
      }

The casing of User names (Must match exactly) and Group/Feed names (must be lower case) tripped me up yet again. The “automagic” provisioning of feeds does make setting up small scale systems easier, though I’m not certain how well it would scale.

Adafruit MQTT Cloud to Device Messaging

After getting MQ Telemetry Transport (MQTT) Device to Cloud (D2C) messaging working for AdaFruit.IO I have also got Cloud to Device (C2D) messaging working as well.

The MQTT broker, username, API key, client ID, optional group name (to keep MQTT aligned with REST API terminology), command topic and feed name are command line options.

The Adafruit IO MQTT documentation suggests an approach for naming topics which allows a bit more structure for feed (D2C and C2D) names than the REST API (which only does D2C).

class Program
{
	private static IMqttClient mqttClient = null;
	private static IMqttClientOptions mqttOptions = null;
	private static string server;
	private static string username;
	private static string password;
	private static string clientId;
	private static string commandTopic;
	private static string groupname;
	private static string feedname;

	static void Main(string[] args)
	{
		MqttFactory factory = new MqttFactory();
		mqttClient = factory.CreateMqttClient();

		if ((args.Length != 6) && (args.Length != 7))
		{
			Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID] [CommandTopic] [GroupName] [FeedName]");
			Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID] [CommandTopic] [FeedName]");
			Console.WriteLine("Press <enter> to exit");
			Console.ReadLine();
			return;
		}

		server = args[0];
		username = args[1];
		password = args[2];
		clientId = args[3];
		commandTopic = args[4];
		if (args.Length == 6)
		{
			feedname = args[5].ToLower();
			Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} CommandTopic:{commandTopic} Feedname:{feedname}");
		}

		if (args.Length == 7)
		{
			groupname = args[5].ToLower();
			feedname = args[6].ToLower();
			Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} CommandTopic:{commandTopic} Groupname:{groupname} Feedname:{feedname}");
		}

		mqttOptions = new MqttClientOptionsBuilder()
			.WithTcpServer(server)
			.WithCredentials(username, password)
			.WithClientId(clientId)
			.WithTls()
			.Build();

		mqttClient.Disconnected += MqttClient_Disconnected;
		mqttClient.ConnectAsync(mqttOptions).Wait();
		mqttClient.ApplicationMessageReceived += MqttClient_ApplicationMessageReceived;

		// Adafruit.IO format for topics which are called feeds
		string topic = string.Empty;

		if (args.Length == 6)
		{
			topic = $"{username}/feeds/{feedname}";				
		}

		if (args.Length == 7)
		{
			topic = $"{username}/feeds/{groupname}.{feedname}";
		}

		mqttClient.SubscribeAsync(commandTopic, MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce).GetAwaiter().GetResult();

		while (true)
		{
			string value = "22." + DateTime.UtcNow.Millisecond.ToString();
			Console.WriteLine($"Topic:{topic} Value:{value}");

			var message = new MqttApplicationMessageBuilder()
				.WithTopic(topic)
				.WithPayload(value)
				.WithQualityOfServiceLevel(MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce)
			.WithRetainFlag()
			.Build();

			Console.WriteLine("PublishAsync start");
			mqttClient.PublishAsync(message).Wait();
			Console.WriteLine("PublishAsync finish");

			Thread.Sleep(30100);
		}
	}

	private static void MqttClient_ApplicationMessageReceived(object sender, MqttApplicationMessageReceivedEventArgs e)
	{
		Console.WriteLine($"ClientId:{e.ClientId} Topic:{e.ApplicationMessage.Topic} Payload:{e.ApplicationMessage.ConvertPayloadToString()}");
	}

	private static async void MqttClient_Disconnected(object sender, MqttClientDisconnectedEventArgs e)
	{
		Debug.WriteLine("Disconnected");
		await Task.Delay(TimeSpan.FromSeconds(5));

		try
		{
			await mqttClient.ConnectAsync(mqttOptions);
		}
		catch (Exception ex)
		{
			Debug.WriteLine("Reconnect failed {0}", ex.Message);
		}
	}
}

I configured a slider on the dashboard for my home called “setpoint” (yet again I was tripped up “automatically” camel casing the name because I’m a C# developer) which my MQTT client subscribed to.

AdaFruit.IO Home monitoring dashboard
setpoint feed configuration

After figuring out the format of the command topic I found that when the slider was moved the MQTT client subscription event fired reliably.

AdaFruit Client showing the setpoint value change notifications

Overall the process went pretty well, though the manual configuration of the subscriptions to AdaFruit.IO feeds could become a bit of a problem at scale.

Adafruit MQTT with MQTTnet

Before building the Message Queue Telemetry Transport(MQTT) gateway I built a proof of concept(PoC) .Net core console application. This was to confirm that I could connect to the Adafruit.IO MQTT broker and format the topic (with and without group name) and payload correctly. The Adafruit IO MQTT documentation suggests an approach for naming topics which allows a bit more structure for feed names than the REST API.

The MQTT broker, username, API key, client ID, optional group name (to keep MQTT aligned with REST API terminology) and feed name are command line options.

class Program
{
	private static IMqttClient mqttClient = null;
	private static IMqttClientOptions mqttOptions = null;
	private static string server;
	private static string username;
	private static string password;
	private static string clientId;
	private static string groupname;
	private static string feedname;

	static void Main(string[] args)
	{
		MqttFactory factory = new MqttFactory();
		mqttClient = factory.CreateMqttClient();

		if ((args.Length != 5) && (args.Length != 6))
		{
			Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID] [GroupName] [FeedName]");
			Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID] [FeedName]");
			Console.WriteLine("Press <enter> to exit");
			Console.ReadLine();
			return;
		}

		server = args[0];
		username = args[1];
		password = args[2];
		clientId = args[3];
		if (args.Length == 5)
		{
			feedname = args[4].ToLower();
			Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} Feedname:{feedname}");
		}

		if (args.Length == 6)
		{
			groupname = args[4].ToLower();
			feedname = args[5].ToLower();
			Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} Groupname:{groupname} Feedname:{feedname}");
		}

		mqttOptions = new MqttClientOptionsBuilder()
			.WithTcpServer(server)
			.WithCredentials(username, password)
			.WithClientId(clientId)
			.WithTls()
			.Build();

		mqttClient.Disconnected += MqttClient_Disconnected;
		mqttClient.ConnectAsync(mqttOptions).Wait();

		// Adafruit.IO format for topics which are called feeds
		string topic = string.Empty;

		if (args.Length == 5)
		{
			topic = $"{args[1]}/feeds/{feedname}";
		}

		if (args.Length == 6)
		{
			topic = $"{args[1]}/feeds/{groupname}.{feedname}";
		}

		while (true)
		{
			string value = "22." + DateTime.UtcNow.Millisecond.ToString();
			Console.WriteLine($"Topic:{topic} Value:{value}");

			var message = new MqttApplicationMessageBuilder()
				.WithTopic(topic)
				.WithPayload(value)
				.WithQualityOfServiceLevel(MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce)
				.WithRetainFlag()
				.Build();

			Console.WriteLine("PublishAsync start");
			mqttClient.PublishAsync(message).Wait();
			Console.WriteLine("PublishAsync finish");

			Thread.Sleep(30100);
		}
	}

	private static async void MqttClient_Disconnected(object sender, MqttClientDisconnectedEventArgs e)
	{
		Debug.WriteLine("Disconnected");
		await Task.Delay(TimeSpan.FromSeconds(5));

		try
		{
			await mqttClient.ConnectAsync(mqttOptions);
		}
		catch (Exception ex)
		{
			Debug.WriteLine("Reconnect failed {0}", ex.Message);
		}
	}
}

For this PoC I used the MQTTnet package which is available via NuGet. It appeared to be reasonably well supported and has had recent updates.

Overall the process went pretty well, I found that looking at the topic names in the Adafruit IO feed setup screens helped a lot. A couple of times I was tripped up by mixed case in my text fields.

.Net Core 2 client with group name
Adafruit IO feed setup with group name
Console client without group name
Adafruit IO feed setup without group name

I am also going to try building some clients with the Eclipse Paho project .net client so I can compare a couple of different libraries.

Adafruit.IO nRF24L01 Windows 10 IoT Core Field Gateway with BorosRF2

A couple of BorosRF2 Dual nRF24L01 Hats arrived earlier in the week. After some testing with my nRF24L01 Test application I have added compile-time configuration options for the two nRF24L01 sockets to my Adafruit.IO nRF24L01 Field Gateway.

Boros RF2 with Dual nRF24L01 devices
public sealed class StartupTask : IBackgroundTask
{
   private const string ConfigurationFilename = "config.json";

   private const byte MessageHeaderPosition = 0;
   private const byte MessageHeaderLength = 1;

   // nRF24 Hardware interface configuration
#if CEECH_NRF24L01P_SHIELD
   private const byte RF24ModuleChipEnablePin = 25;
   private const byte RF24ModuleChipSelectPin = 0;
   private const byte RF24ModuleInterruptPin = 17;
#endif

#if BOROS_RF2_SHIELD_RADIO_0
   private const byte RF24ModuleChipEnablePin = 24;
   private const byte RF24ModuleChipSelectPin = 0;
   private const byte RF24ModuleInterruptPin = 27;
#endif

#if BOROS_RF2_SHIELD_RADIO_1
   private const byte RF24ModuleChipEnablePin = 25;
   private const byte RF24ModuleChipSelectPin = 1;
   private const byte RF24ModuleInterruptPin = 22;
#endif

private readonly LoggingChannel loggingChannel = new LoggingChannel("devMobile AdaFruit.IO nRF24L01 Field Gateway", null, new Guid("4bd2826e-54a1-4ba9-bf63-92b73ea1ac4a"));
private readonly RF24 rf24 = new RF24();

For this initial version only one nRF24L01 device socket active at a time is supported.

Windows 10 IoT Core Field Gateways “less is more”

After looking back at the technical support interactions for my Azure IoT Hubs Windows 10 IoT Core Field Gateway & AdaFruit.IO LoRa Windows 10 IoT Core Field Gateway I think removing a “feature” might make it easier for first time users.

In an early version of the software I used to provide a sample configuration JSON file in the associated GitHub repository. Users had to download this file to a computer, update it with their Azure IOT Hub or Azure IoT Central connection string or AdafruitIO APIKey , frequency and device address, then upload to the field gateway.

In a later version of the software I added code which created an empty configuration file with defaults for all settings, many of which were a distraction as the majority of users would never change them.

More settings meant there was more scope for users to change settings which broke the device samples and the gateway.

I have removed the code to generate the full configuration file (starting with Azure IOT Hub field gateway) and included a sample configuration file with the minimum required settings in the GitHub repositories and installers.

I am assuming that if a user wants to change advanced settings they can look at the code and/or documentation and figure out the setting names and valid values.

The new sample configuration file for a Azure IoT Hub telemetry only gateway is

{
  "AzureIoTHubDeviceConnectionString": "Azure IOT Hub connection string",
  "AzureIoTHubTransportType": "amqp",
  "SensorIDIsDeviceIDSensorID": false,
  "Address": "Device address",
  "Frequency": 915000000.0
}

The prebuilt installers available on GitHub post version 1.0.13.0 (Azure IoT Hub) and 1.0.5.0 (Adafruit.IO) will implement this model.

SparkFun Pro RF – LoRa, 915MHz Payload Addressing Client

Last week a package arrived from NiceGear with a SparkFun Pro RF – LoRa, 915MHz and some cables. With this gear I have built yet another client for my Azure IoT Hub and AdaFruit.IOLoRa Field Gateways.

Now that the device is running well, I’ll look at reducing power consumption and splitting the the payload packing code into a library. Also noticed an extra “,” on the end of a message so need to come up with a better way of doing the payload packing.

/*
  Copyright ® 2018 December devMobile Software, All Rights Reserved

  THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY
  KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
  IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR
  PURPOSE.

  You can do what you want with this code, acknowledgment would be nice.

  http://www.devmobile.co.nz

*/
#include <stdlib.h>

#include <LoRa.h>
#include <avr/dtostrf.h>
#include "DHT.h"

#define DEBUG
//#define DEBUG_TELEMETRY
//#define DEBUG_LORA

// LoRa field gateway configuration (these settings must match your field gateway)
const char FieldGatewayAddress[] = {"LoRaIoT1"};
const char DeviceAddress[] = {"SparkFunX1"};
const float FieldGatewayFrequency =  915000000.0;
const byte FieldGatewaySyncWord = 0x12 ;

// Payload configuration
const int InterruptPin = 12;
const int ChipSelectPin = 6;

// LoRa radio payload configuration
const byte SensorIdValueSeperator = ' ' ;
const byte SensorReadingSeperator = ',' ;
const int LoopSleepDelaySeconds = 10 ;

const byte PayloadSizeMaximum = 64 ;
byte payload[PayloadSizeMaximum];
byte payloadLength = 0 ;

#define DHTPIN 4     // what digital pin we're connected to

// Uncomment whatever type you're using!
//#define DHTTYPE DHT11   // DHT 11
#define DHTTYPE DHT22   // DHT 22  (AM2302), AM2321
//#define DHTTYPE DHT21   // DHT 21 (AM2301)

DHT dht(DHTPIN, DHTTYPE);


void setup()
{
  SerialUSB.begin(9600);
#ifdef DEBUG
  while (!SerialUSB);
#endif
  SerialUSB.println("Setup called");

  SerialUSB.println("LoRa setup start");
  
  // override the default chip select and reset pins
  LoRa.setPins(InterruptPin, ChipSelectPin); 
  if (!LoRa.begin(FieldGatewayFrequency))
  {
    SerialUSB.println("LoRa begin failed");
    while (true); // Drop into endless loop requiring restart
  }

  // Need to do this so field gateways pays attention to messsages from this device
  LoRa.enableCrc();
  LoRa.setSyncWord(FieldGatewaySyncWord);

#ifdef DEBUG_LORA
  LoRa.dumpRegisters(SerialUSB);
#endif
  SerialUSB.println("LoRa Setup done.");

  // Configure the Seeedstudio TH02 temperature & humidity sensor
  SerialUSB.println("DHT setup start");
  dht.begin();
  delay(100);
  SerialUSB.println("DHT setup done");

  PayloadHeader((byte*)FieldGatewayAddress,strlen(FieldGatewayAddress), (byte*)DeviceAddress, strlen(DeviceAddress));

  SerialUSB.println("Setup done");
  SerialUSB.println();
}


void loop()
{
  float temperature = 17.2;
  float humidity = 75.0;

  SerialUSB.println("Loop called");

  PayloadReset();

  // Read the temperature & humidity & battery voltage values then display nicely
  temperature = dht.readTemperature();
  humidity = dht.readHumidity();
  if (isnan(humidity) || isnan(temperature)) 
  {
    SerialUSB.println("Failed to read from DHT sensor!");
    return;
  } 
   
  SerialUSB.print("T:");
  SerialUSB.print( temperature, 1 ) ;
  SerialUSB.println( "C " ) ;
  PayloadAdd( "T", temperature, 1);

  SerialUSB.print("H:");
  SerialUSB.print( humidity, 0 ) ;
  SerialUSB.println( "% " ) ;
  PayloadAdd( "H", humidity, 0) ;

#ifdef DEBUG_TELEMETRY
  SerialUSB.println();
  SerialUSB.print( "RFM9X/SX127X Payload length:");
  SerialUSB.print( payloadLength );
  SerialUSB.println( " bytes" );
#endif

  LoRa.beginPacket();
  LoRa.write( payload, payloadLength );
  LoRa.endPacket();

  SerialUSB.println("Loop done");
  SerialUSB.println();
  delay(LoopSleepDelaySeconds * 1000l);
}


void PayloadHeader( byte *to, byte toAddressLength, byte *from, byte fromAddressLength)
{
  byte addressesLength = toAddressLength + fromAddressLength ;

#ifdef DEBUG_TELEMETRY
  SerialUSB.println("PayloadHeader- ");
  SerialUSB.print( "To Address len:");
  SerialUSB.print( toAddressLength );
  SerialUSB.print( " From Address len:");
  SerialUSB.print( fromAddressLength );
  SerialUSB.print( " Addresses length:");
  SerialUSB.print( addressesLength );
  SerialUSB.println( );
#endif

  payloadLength = 0 ;

  // prepare the payload header with "To" Address length (top nibble) and "From" address length (bottom nibble)
  payload[payloadLength] = (toAddressLength << 4) | fromAddressLength ;
  payloadLength += 1;

  // Copy the "To" address into payload
  memcpy(&payload[payloadLength], to, toAddressLength);
  payloadLength += toAddressLength ;

  // Copy the "From" into payload
  memcpy(&payload[payloadLength], from, fromAddressLength);
  payloadLength += fromAddressLength ;
}


void PayloadAdd( char *sensorId, float value, byte decimalPlaces)
{
  byte sensorIdLength = strlen( sensorId ) ;

#ifdef DEBUG_TELEMETRY
  SerialUSB.println("PayloadAdd-float ");
  SerialUSB.print( "SensorId:");
  SerialUSB.print( sensorId );
  SerialUSB.print( " sensorIdLen:");
  SerialUSB.print( sensorIdLength );
  SerialUSB.print( " Value:");
  SerialUSB.print( value, decimalPlaces );
  SerialUSB.print( " payloadLength:");
  SerialUSB.print( payloadLength);
#endif

  memcpy( &payload[payloadLength], sensorId,  sensorIdLength) ;
  payloadLength += sensorIdLength ;
  payload[ payloadLength] = SensorIdValueSeperator;
  payloadLength += 1 ;
  payloadLength += strlen( dtostrf(value, -1, decimalPlaces, (char *)&payload[payloadLength]));
  payload[ payloadLength] = SensorReadingSeperator;
  payloadLength += 1 ;
  
#ifdef DEBUG_TELEMETRY
  SerialUSB.print( " payloadLength:");
  SerialUSB.print( payloadLength);
  SerialUSB.println( );
#endif
}


void PayloadAdd( char *sensorId, int value )
{
  byte sensorIdLength = strlen( sensorId ) ;

#ifdef DEBUG_TELEMETRY
  SerialUSB.println("PayloadAdd-int ");
  SerialUSB.print( "SensorId:");
  SerialUSB.print( sensorId );
  SerialUSB.print( " sensorIdLen:");
  SerialUSB.print( sensorIdLength );
  SerialUSB.print( " Value:");
  SerialUSB.print( value );
  SerialUSB.print( " payloadLength:");
  SerialUSB.print( payloadLength);
#endif  

  memcpy( &payload[payloadLength], sensorId,  sensorIdLength) ;
  payloadLength += sensorIdLength ;
  payload[ payloadLength] = SensorIdValueSeperator;
  payloadLength += 1 ;
  payloadLength += strlen( itoa( value,(char *)&payload[payloadLength],10));
  payload[ payloadLength] = SensorReadingSeperator;
  payloadLength += 1 ;
  
#ifdef DEBUG_TELEMETRY
  SerialUSB.print( " payloadLength:");
  SerialUSB.print( payloadLength);
  SerialUSB.println( );
#endif
}


void PayloadAdd( char *sensorId, unsigned int value )
{
  byte sensorIdLength = strlen( sensorId ) ;

#ifdef DEBUG_TELEMETRY
  SerialUSB.println("PayloadAdd-unsigned int ");
  SerialUSB.print( "SensorId:");
  SerialUSB.print( sensorId );
  SerialUSB.print( " sensorIdLen:");
  SerialUSB.print( sensorIdLength );
  SerialUSB.print( " Value:");
  SerialUSB.print( value );
  SerialUSB.print( " payloadLength:");
  SerialUSB.print( payloadLength);
#endif  

  memcpy( &payload[payloadLength], sensorId,  sensorIdLength) ;
  payloadLength += sensorIdLength ;
  payload[ payloadLength] = SensorIdValueSeperator;
  payloadLength += 1 ;
  payloadLength += strlen( utoa( value,(char *)&payload[payloadLength],10));
  payload[ payloadLength] = SensorReadingSeperator;
  payloadLength += 1 ;

#ifdef DEBUG_TELEMETRY
  SerialUSB.print( " payloadLength:");
  SerialUSB.print( payloadLength);
  SerialUSB.println( );
#endif
}


void PayloadReset()
{
  byte fromAddressLength = payload[0] & 0xf ;
  byte toAddressLength = payload[0] >> 4 ;
  byte addressesLength = toAddressLength + fromAddressLength ;

  payloadLength = addressesLength + 1;

#ifdef DEBUG_TELEMETRY
  SerialUSB.println("PayloadReset- ");
  SerialUSB.print( "To Address len:");
  SerialUSB.print( toAddressLength );
  SerialUSB.print( " From Address len:");
  SerialUSB.print( fromAddressLength );
  SerialUSB.print( " Addresses length:");
  SerialUSB.print( addressesLength );
  SerialUSB.println( );
#endif
}
  • SparkFun Pro RF – LoRa, 915MHz USD29.95 NZD49
  • Grove – Temperature & Humidity Sensor Pro (AM2302) USD9.90
  • Seeedstudio 4 pin Male Jumper to Grove 4 pin Conversion Cable USD2.90

Adafruit.IO LoRa Field Gateway BCD Addressing

After some testing with more client devices, especially the Easy Sensors Arduino Nano radio shield RFM69/95 or NRF24L01+ I have decided to move to non text addresses for devices and the LoRa field gateway.

THIS IS A BREAKING CHANGE

The unique identifier provided by the SHA204A crypto and authentication chip on the EasySensors shield highlighted this issue. The Binary Coded Decimal(BCD) version of the 72 bit identifier was too long to fit in the from address.

My later Arduino based sample clients have some helper functions to populate the message header, add values, and prepare the message payload for reuse.

On the server side I have added code to log the build version and Raspbery PI shield type

// Log the Application build, shield information etc.
LoggingFields appllicationBuildInformation = new LoggingFields();
#if DRAGINO
   appllicationBuildInformation.AddString("Shield", "DraginoLoRaGPSHat");
#endif
…
#if UPUTRONICS_RPIPLUS_CS1
   appllicationBuildInformation.AddString("Shield", "UputronicsPiPlusLoRaExpansionBoardCS1");
#endif
appllicationBuildInformation.AddString("Timezone", TimeZoneSettings.CurrentTimeZoneDisplayName);
appllicationBuildInformation.AddString("OSVersion", Environment.OSVersion.VersionString);
appllicationBuildInformation.AddString("MachineName", Environment.MachineName);

// This is from the application manifest 
Package package = Package.Current;
PackageId packageId = package.Id;
PackageVersion version = packageId.Version;

appllicationBuildInformation.AddString("ApplicationVersion", string.Format($"{version.Major}.{version.Minor}.{version.Build}.{version.Revision}"));
this.loggingChannel.LogEvent("Application starting", appllicationBuildInformation, LoggingLevel.Information);

Then when the message payload is populated the from address byte array is converted to BCD

private async void Rfm9XDevice_OnReceive(object sender, Rfm9XDevice.OnDataReceivedEventArgs e)
{
   string addressBcdText;
   string messageBcdText;
   string messageText = "";
   char[] sensorReadingSeparator = new char[] { ',' };
   char[] sensorIdAndValueSeparator = new char[] { ' ' };

   addressBcdText = BitConverter.ToString(e.Address);

   messageBcdText = BitConverter.ToString(e.Data);
   try
   {
      messageText = UTF8Encoding.UTF8.GetString(e.Data);
   }
   catch (Exception)
   {
      this.loggingChannel.LogMessage("Failure converting payload to text", LoggingLevel.Error);
   return;
   }

#if DEBUG
    Debug.WriteLine(@"{0:HH:mm:ss}-RX From {1} PacketSnr {2:0.0} Packet RSSI {3}dBm RSSI {4}dBm = {5} byte message ""{6}""", DateTime.Now, addressBcdText, e.PacketSnr, e.PacketRssi, e.Rssi, e.Data.Length, messageText);
#endif
   LoggingFields messagePayload = new LoggingFields();
   messagePayload.AddInt32("AddressLength", e.Address.Length);
   messagePayload.AddString("Address-BCD", addressBcdText);
   messagePayload.AddInt32("Message-Length", e.Data.Length);
   messagePayload.AddString("Message-BCD", messageBcdText);
   messagePayload.AddString("Nessage-Unicode", messageText);
   messagePayload.AddDouble("Packet SNR", e.PacketSnr);
   messagePayload.AddInt32("Packet RSSI", e.PacketRssi);
   messagePayload.AddInt32("RSSI", e.Rssi);
   this.loggingChannel.LogEvent("Message Data", messagePayload, LoggingLevel.Verbose);

			
   // Check the address is not to short/long 
   if (e.Address.Length < AddressLengthMinimum)
   {
      this.loggingChannel.LogMessage("From address too short", LoggingLevel.Warning);
      return;
   }

   if (e.Address.Length > MessageLengthMaximum)
   {
      this.loggingChannel.LogMessage("From address too long", LoggingLevel.Warning);
      return;
   }

   // Check the payload is not too short/long 
   if (e.Data.Length < MessageLengthMinimum)
   {
      this.loggingChannel.LogMessage("Message too short to contain any data", LoggingLevel.Warning);
      return;
   }

   if (e.Data.Length > MessageLengthMaximum)
   {
      this.loggingChannel.LogMessage("Message too long to contain valid data", LoggingLevel.Warning);
      return;
   }

   // Adafruit IO is case sensitive & only does lower case ?
   string deviceId = addressBcdText.ToLower();

   // Chop up the CSV text payload
   string[] sensorReadings = messageText.Split(sensorReadingSeparator, StringSplitOptions.RemoveEmptyEntries);
   if (sensorReadings.Length == 0)
   {
      this.loggingChannel.LogMessage("Payload contains no sensor readings", LoggingLevel.Warning);
      return;
   }

   Group_feed_data groupFeedData = new Group_feed_data();

   LoggingFields sensorData = new LoggingFields();
   sensorData.AddString("DeviceID", deviceId);

   // Chop up each sensor reading into an ID & value
   foreach (string sensorReading in sensorReadings)
   {
      string[] sensorIdAndValue = sensorReading.Split(sensorIdAndValueSeparator, StringSplitOptions.RemoveEmptyEntries);

      // Check that there is an id & value
      if (sensorIdAndValue.Length != 2)
      {
         this.loggingChannel.LogMessage("Sensor reading invalid format", LoggingLevel.Warning);
         return;
      }

      string sensorId = sensorIdAndValue[0].ToLower();
      string value = sensorIdAndValue[1];

      // Construct the sensor ID from SensordeviceID & Value ID
      groupFeedData.Feeds.Add(new Anonymous2() { Key = string.Format("{0}{1}", deviceId, sensorId), Value = value });

      sensorData.AddString(sensorId, value);

      Debug.WriteLine(" Sensor {0}{1} Value {2}", deviceId, sensorId, value);
   }
   this.loggingChannel.LogEvent("Sensor readings", sensorData, LoggingLevel.Verbose);

   try
   {
      Debug.WriteLine(" CreateGroupDataAsync start");
      await this.adaFruitIOClient.CreateGroupDataAsync(this.applicationSettings.AdaFruitIOUserName,
this.applicationSettings.AdaFruitIOGroupName.ToLower(), groupFeedData);
      Debug.WriteLine(" CreateGroupDataAsync finish");
   }
   catch (Exception ex)
   {
      Debug.WriteLine(" CreateGroupDataAsync failed {0}", ex.Message);
				this.loggingChannel.LogMessage("CreateGroupDataAsync failed " + ex.Message, LoggingLevel.Error);
   }
}
AfaFruit.IO Data Display

This does mean longer field names but I usually copy n paste them from the Arduino serial monitor or the Event Tracing For Windows (ETW) logging.

AdaFruit.IO Field gateway ETW Logging