TTI V3 Connector Azure Storage Queues Paused

After running my The Things Industries(TTI) V3 HTTPStorageQueueOutput application for a week I think there are some problems with my approach so I have paused development while I build another HTTPTrigger Azure Functions based Proof of Concept(PoC).

The HTTPTrigger and Azure Storage Queue OutputBinding based code which inserts messages into an Azure Storage Queue was minimal

[StorageAccount("AzureWebJobsStorage")]
public static class Webhooks
{
	[Function("Uplink")]
	public static async Task<HttpTriggerUplinkOutputBindingType> Uplink([HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req, FunctionContext context)
	{
		var logger = context.GetLogger("UplinkMessage");

		logger.LogInformation("Uplink processed");
			
		var response = req.CreateResponse(HttpStatusCode.OK);

		return new HttpTriggerUplinkOutputBindingType()
		{
			Name = await req.ReadAsStringAsync(),
			HttpReponse = response
		};
	}
}

With Azure Storage Explorer I could inspect uplink, queued, sent, and acknowledgment(ACK) messages. It was difficult to generate failed and Negative Acknowledgement (Nack) and failed messages

Azure Storage Explorer displaying Uplink messages
Azure Storage Explorer displaying queued messages
Azure Storage Explorer displaying sent messages
Azure Storage Explorer Displaying Ack messages

After some experimentation I realised that I had forgotten that the order of message processing was important e.g. a TTI Queued message should be processed before the associated Ack. This could (and did happen) because I had a queue for each message type and in addition the Azure Queue Storage trigger binding would use parallel execution to process backlogs of messages. My approach caused issues with both intra and inter queue message ordering

TTI V3 Connector Azure Storage Queues

The first Proof of Concept(PoC) for my updated The Things Industries(TTI) V3 Webhooks Integration was to explore the use of Azure Functions to securely ingest webhook calls. The aim was to have uplink and downlink message progress message payloads written to Azure Storage Queues with output bindings ready for processing.

namespace devMobile.IoT.TheThingsIndustries.HttpInputStorageQueueOutput
{
	using System.Net;
	using System.Threading.Tasks;

	using Microsoft.Azure.Functions.Worker;
	using Microsoft.Azure.Functions.Worker.Http;
	using Microsoft.Azure.WebJobs;
	using Microsoft.Extensions.Logging;


	[StorageAccount("AzureWebJobsStorage")]
	public static class Webhooks
	{
		[Function("Uplink")]
		public static async Task<HttpTriggerUplinkOutputBindingType> Uplink([HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req, FunctionContext context)
		{
			var logger = context.GetLogger("UplinkMessage");

			logger.LogInformation("Uplink processed");
			
			var response = req.CreateResponse(HttpStatusCode.OK);

			return new HttpTriggerUplinkOutputBindingType()
			{
				Name = await req.ReadAsStringAsync(),
				HttpReponse = response
			};
		}

		public class HttpTriggerUplinkOutputBindingType
		{
			[QueueOutput("uplink")]
			public string Name { get; set; }

			public HttpResponseData HttpReponse { get; set; }
		}

...

		[Function("Failed")]
		public static async Task<HttpTriggerFailedOutputBindingType> Failed([HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req, FunctionContext context)
		{
			var logger = context.GetLogger("Failed");

			logger.LogInformation("Failed procssed");

			var response = req.CreateResponse(HttpStatusCode.OK);

			return new HttpTriggerFailedOutputBindingType()
			{
				Name = await req.ReadAsStringAsync(),
				HttpReponse = response
			};
		}

		public class HttpTriggerFailedOutputBindingType
		{
			[QueueOutput("failed")]
			public string Name { get; set; }

			public HttpResponseData HttpReponse { get; set; }
		}
	}
}

After some initial problems with the use of Azure Storage Queue output bindings to insert messages into the ack, nak, failed, queued, and uplink Azure Storage Queues I found it didn’t take much code and worked reliably on my desktop.

Azure Functions Desktop Development environment running my functions

I used Telerik Fiddler with some sample payloads to test my application.

Telerik Fiddler Request Composer “posting” sample message to desktop endpoint

Once the functions were running reliably on my desktop, I created an Azure Service Plan, deployed the code, then generated an API Key for securing my HTTPTrigger endpoints.

Azure Functions Host Key configuration dialog

I then added a TTI Webhook Integration to my TTI SeeduinoLoRaWAN application, manually configured the endpoint, enabled the different messages I wanted to process and set the x-functions-key header.

TTI Application Webhook configuration

After a short delay I could see messages in the message uplink queue with Azure Storage Explorer

Azure Storage Explorer displaying content of my uplink queue

Building a new version of my TTIV3 Azure IoT connector is a useful learning exercise but I’m still deciding whether is it worth the effort as TTI has one now?

.NET Core RAK811 LoRaWAN library Part3

The massive refactor

After refactoring my RAK3172 device library I have applied a similar approach to code on my RAK811 device library. My test-rig is a RaspberryPI 3B with a PI Supply RAK811 pHat and external antenna.

PI Supply RAK811 LoRaWAN pHat

In the new code a Thread reads lines of text from the SerialPort and processes them, checking for command responses, failures and downlink messages.

Unlike most of the devices I have worked with the RAK811 Join and Send commands are synchronous so return once the process has completed. The RAK811 responses also have quite a few empty, null prefixed or null suffixed lines which is a bit odd.

public void SerialPortProcessor()
{
	string line;

	while (CommandProcessResponses)
	{
		try
		{
#if DIAGNOSTICS
			Debug.WriteLine($" {DateTime.UtcNow:hh:mm:ss} ReadLine before");
#endif
			line = SerialDevice.ReadLine().Trim('\0').Trim();
#if DIAGNOSTICS
			Debug.WriteLine($" {DateTime.UtcNow:hh:mm:ss} ReadLine after:{line}");
#endif
			// consume empty lines
			if (String.IsNullOrWhiteSpace(line))
			{
				continue;
			}

			// Consume the response from set work mode
			if (line.StartsWith("?LoRa (R)") || line.StartsWith("RAK811 ") || line.StartsWith("UART1 ") || line.StartsWith("UART3 ") || line.StartsWith("LoRa work mode"))
			{
				continue;
			}

			// See if device successfully joined network
			if (line.StartsWith("OK Join Success"))
			{
				OnJoinCompletion?.Invoke(true);

				CommandResponseExpectedEvent.Set();

				continue;
			}

			if (line.StartsWith("at+recv="))
			{
				string[] payloadFields = line.Split("=,:".ToCharArray());

				byte port = byte.Parse(payloadFields[1]);
				int rssi = int.Parse(payloadFields[2]);
				int snr = int.Parse(payloadFields[3]);
				int length = int.Parse(payloadFields[4]);

				if (this.OnMessageConfirmation != null)
				{
					OnMessageConfirmation?.Invoke(rssi, snr);
				}
				if (length > 0)
				{
					string payload = payloadFields[5];

					if (this.OnReceiveMessage != null)
					{
						OnReceiveMessage.Invoke(port, rssi, snr, payload);
					}
				}
				continue;
			}

			switch (line)
			{
				case "OK":
				case "Initialization OK":
				case "OK Wake Up":
				case "OK Sleep":
					CommandResult = Result.Success;
					break;

				case "ERROR: 1":
					CommandResult = Result.ATCommandUnsuported;
					break;
				case "ERROR: 2":
					CommandResult = Result.ATCommandInvalidParameter;
					break;
				case "ERROR: 3": //There is an error when reading or writing flash.
				case "ERROR: 4": //There is an error when reading or writing through IIC.
					CommandResult = Result.ErrorReadingOrWritingFlash;
					break;
				case "ERROR: 5": //There is an error when sending through UART
					CommandResult = Result.ATCommandInvalidParameter;
					break;
				case "ERROR: 41": //The BLE works in an invalid state, so that it can’t be operated.
					CommandResult = Result.ResponseInvalid;
					break;
				case "ERROR: 80":
					CommandResult = Result.LoRaBusy;
					break;
				case "ERROR: 81":
					CommandResult = Result.LoRaServiceIsUnknown;
					break;
				case "ERROR: 82":
					CommandResult = Result.LoRaParameterInvalid;
					break;
				case "ERROR: 83":
					CommandResult = Result.LoRaFrequencyInvalid;
					break;
				case "ERROR: 84":
					CommandResult = Result.LoRaDataRateInvalid;
					break;
				case "ERROR: 85":
					CommandResult = Result.LoRaFrequencyAndDataRateInvalid;
					break;
				case "ERROR: 86":
					CommandResult = Result.LoRaDeviceNotJoinedNetwork;
					break;
				case "ERROR: 87":
					CommandResult = Result.LoRaPacketToLong;
					break;
				case "ERROR: 88":
					CommandResult = Result.LoRaServiceIsClosedByServer;
					break;
				case "ERROR: 89":
					CommandResult = Result.LoRaRegionUnsupported;
					break;
				case "ERROR: 90":
					CommandResult = Result.LoRaDutyCycleRestricted;
					break;
				case "ERROR: 91":
					CommandResult = Result.LoRaNoValidChannelFound;
					break;
				case "ERROR: 92":
					CommandResult = Result.LoRaNoFreeChannelFound;
					break;
				case "ERROR: 93":
					CommandResult = Result.StatusIsError;
					break;
				case "ERROR: 94":
					CommandResult = Result.LoRaTransmitTimeout;
					break;
				case "ERROR: 95":
					CommandResult = Result.LoRaRX1Timeout;
					break;
				case "ERROR: 96":
					CommandResult = Result.LoRaRX2Timeout;
					break;
				case "ERROR: 97":
					CommandResult = Result.LoRaRX1ReceiveError;
					break;
				case "ERROR: 98":
					CommandResult = Result.LoRaRX2ReceiveError;
					break;
				case "ERROR: 99":
					CommandResult = Result.LoRaJoinFailed;
					break;
				case "ERROR: 100":
					CommandResult = Result.LoRaDownlinkRepeated;
					break;
				case "ERROR: 101":
					CommandResult = Result.LoRaPayloadSizeNotValidForDataRate;
					break;
				case "ERROR: 102":
					CommandResult = Result.LoRaTooManyDownlinkFramesLost;
					break;
				case "ERROR: 103":
					CommandResult = Result.LoRaAddressFail;
					break;
				case "ERROR: 104":
					CommandResult = Result.LoRaMicVerifyError;
					break;
				default:
					CommandResult = Result.ResponseInvalid;
					break;
			}
		}
		catch (TimeoutException)
		{
			// Intentionally ignored, not certain this is a good idea
		}

		CommandResponseExpectedEvent.Set();
	}
}

After a lot of testing I think my thread based approach works reliably. Initially, I was having some signal strength issues because I had forgotten to configure the external antenna. I need to add some validation to the metrics and payload field unpacking (though I’m not certain what todo if they are the wrong format).

TTN V3 Connector Revisited

Earlier in the year I built Things Network(TTN) V2 and V3 connectors and after using these in production applications I have learnt a lot about what I had got wrong, less wrong and what I had got right.

Using a TTN V3 MQTT Application integration wasn’t a great idea. The management of state was very complex. The storage of application keys in a app.settings file made configuration easy but was bad for security.

The use of Azure Key Vault in the TTNV2 connector was a good approach, but the process of creation and updating of the settings needs to be easier.

Using TTN device registry as the “single source of truth” was a good decision as managing the amount of LoRaWAN network, application and device specific configuration in an Azure IoT Hub would be non-trivial.

Using a Webhooks Application Integration like the TTNV2 connector is my preferred approach.

The TTNV2 Connector’s use of Azure Storage Queues was a good idea as they it provide an elastic buffer between the different parts of the application.

The use of Azure Functions to securely ingest webhook calls and write them to Azure Storage Queues with output bindingts should simplify configuration and deployment. The use of Azure Storage Queue input bindings to process messages is the preferred approach.

The TTN V3 processing of JSON uplink messages into a structure that Azure IoT Central could ingest is a required feature

The TTN V2 and V3 support for the Azure Device Provisioning Service(DPS) is a required feature (mandated by Azure IoT Central). The TTN V3 connector support for DTDLV2 is a desirable feature. The DPS implementation worked with Azure IoT Central but I was unable to get the DeviceClient based version working.

Using DPS to pre-provision devices in Azure IoT Hubs and Azure IoT Central by using the TTN Application Registry API then enumerating the TTN applications, then devices needs to be revisited as it was initially slow then became quite complex.

The support for Azure IoT Hub connection strings was a useful feature, but added some complexity. This plus basic Azure IoT Hub DPS support(No Azure IoT Central support) could be implemented in a standalone application which connects via Azure Storage Queue messages.

The processing of Azure IoT Central Basic, and Request commands then translating the payloads so they work with TTN V3 is a required feature. The management of Azure IoT Hub command delivery confirmations (abandon, complete and Reject) is a required feature.

I’m considering building a new TTN V3 connector but is it worth the effort as TTN has one now?

.NET Core RAK3172 LoRaWAN library Part5

The massive refactor

After getting Activation By Personalisation(ABP) and Over The Air Activation(OTAA) working on my RAK3172 test rig I was looking at the code and SerialDataReceivedEventHandler was really ugly.

Raspberry Pi3 with Grove Base Hat and RAK3172 Breakout (using UART2)

After some experimentation in the BreakOutSerial project I decided to reimplement the RAK3172 command processing. In the new code a Thread reads lines of text from the SerialPort and processes them. I have replaced the Join and Send(Confirmed) methods with ones that block only while the command are sent to the RAK3172. Then, when completed the OnJoinCompletion or OnMessagesConfirmation event handlers are called.

private Result SendCommand(string command)
{
	if (command == null)
	{
		throw new ArgumentNullException(nameof(command));
	}

	if (command == string.Empty)
	{
		throw new ArgumentException($"command cannot be empty", nameof(command));
	}

	serialDevice.WriteLine(command);

	this.CommandResponseExpectedEvent.Reset();

	if (!this.CommandResponseExpectedEvent.WaitOne(CommandTimeoutDefaultmSec, false))
	{
		return Result.Timeout;
	}

	return CommandResult;
}

private void SerialPortProcessor()
{
	string line;

	while (CommandProcessResponses)
	{
		try
		{
#if DIAGNOSTICS
			Debug.WriteLine($" {DateTime.UtcNow:hh:mm:ss} ReadLine before");
#endif
			line = serialDevice.ReadLine();
#if DIAGNOSTICS
			Debug.WriteLine($" {DateTime.UtcNow:hh:mm:ss} ReadLine after:{line}");
#endif

			// See if device successfully joined network
			if (line.StartsWith("+EVT:JOINED"))
			{
				OnJoinCompletion?.Invoke(true);

				continue;
			}

			// See if device failed ot join network
			if (line.StartsWith("+EVT:JOIN FAILED"))
			{
				OnJoinCompletion?.Invoke(false);

				continue;
			}

			// Applicable only if confirmed messages enabled 
			if (line.StartsWith("+EVT:SEND CONFIRMED OK"))
			{
				OnMessageConfirmation?.Invoke();

				continue;
			}

			// Check for A/B/C downlink message
			if (line.StartsWith("+EVT:RX_1") || line.StartsWith("+EVT:RX_2") || line.StartsWith("+EVT:RX_3") || line.StartsWith("+EVT:RX_C"))
			{
				// TODO beef up validation, nto certain what todo if borked
				string[] metricsFields= line.Split(' ', ',');

				int rssi = int.Parse(metricsFields[3]);
				int snr = int.Parse(metricsFields[6]);

				line = serialDevice.ReadLine();

#if DIAGNOSTICS
				Debug.WriteLine($" {DateTime.UtcNow:HH:mm:ss} UNICAST :{line}");
#endif
				line = serialDevice.ReadLine();
#if DIAGNOSTICS
				Debug.WriteLine($" {DateTime.UtcNow:HH:mm:ss} Payload:{line}");
#endif
				// TODO beef up validation, nto certain what todo if borked
				string[] payloadFields = line.Split(':');

				byte port = byte.Parse(payloadFields[1]);
				string payload = payloadFields[2];

				OnReceiveMessage?.Invoke(port, rssi, snr, payload);

				continue;
			}

#if DIAGNOSTICS
           Debug.WriteLine($" {DateTime.UtcNow:hh:mm:ss} ReadLine Result");
#endif
			line = serialDevice.ReadLine();
#if DIAGNOSTICS
             Debug.WriteLine($" {DateTime.UtcNow:hh:mm:ss} ReadLine Result:{line}");
#endif
			switch (line)
			{
				case "OK":
					CommandResult = Result.Success;
					break;
				case "AT_ERROR":
					CommandResult = Result.AtError;
					break;
				case "AT_PARAM_ERROR":
					CommandResult = Result.ParameterError;
					break;
				case "AT_BUSY_ERROR":
					CommandResult = Result.BusyError;
					break;
				case "AT_TEST_PARAM_OVERFLOW":
					CommandResult = Result.ParameterOverflow;
					break;
				case "AT_NO_NETWORK_JOINED":
					CommandResult = Result.NotJoined;
					break;
				case "AT_RX_ERROR":
					CommandResult = Result.ReceiveError;
					break;
				case "AT_DUTYCYLE_RESTRICTED":
					CommandResult = Result.DutyCycleRestricted;
					break;
				default:
					CommandResult = Result.Undefined;
					break;
			}

			CommandResponseExpectedEvent.Set();
		}
		catch (TimeoutException)
		{
			// Intentionally ignored, not certain this is a good idea
		}
	}
}

After a lot of testing I think my thread based approach works reliably. I also had to modify the code to shutdown the command processor thread and free any non managed resources.

/// <summary>
/// Ensures unmanaged serial port and thread resources are released in a "responsible" manner.
/// </summary>
public void Dispose()
{
	CommandProcessResponses = false;

	if (CommandResponsesProcessorThread != null)
	{
		CommandResponsesProcessorThread.Join();
		CommandResponsesProcessorThread = null;
	}

	if (serialDevice != null)
	{
		serialDevice.Dispose();
		serialDevice = null;
	}
}

I need to add some validation to the metrics and payload field unpacking (though I’m not certain what todo if they are the wrong format) and review the handling of multi-line event messages.

.NET Core RAK3172 LoRaWAN library Part4

Starting again with Threads

After getting Activation By Personalisation(ABP) and Over The Air Activation(OTAA) working on my RAK3172 test rig I was looking at the code and SerialDataReceivedEventHandler was really ugly.

Raspberry Pi3 with Grove Base Hat and RAK3172 Breakout (using UART2)

After some experimentation in the BreakOutSerial project I decided to reimplement the RAK3172 command processing. In the new code a Thread reads lines of text from the SerialPort and processes them. I have replaced the Join and Send(Confirmed) methods with ones that block only while the command are sent to the RAK3172. Then, when completed the OnJoinCompletion or OnMessagesConfirmation event handlers are called.

private Result SendCommand(string command)
{
   if (command == null)
   {
      throw new ArgumentNullException(nameof(command));
   }

   if (command == string.Empty)
   {
      throw new ArgumentException($"command invalid length cannot be empty", nameof(command));
    }

   serialDevice.ReadTimeout = (int)CommandTimeoutDefault.TotalMilliseconds;
   serialDevice.WriteLine(command);

   this.atExpectedEvent.Reset();

   if (!this.atExpectedEvent.WaitOne((int)CommandTimeoutDefault.TotalMilliseconds, false))
      return Result.Timeout;

   return result;
}

public void SerialPortProcessor()
{
   string line;

   while (true)
   {
      this.serialDevice.ReadTimeout = -1;

      Debug.WriteLine("ReadLine before");
      line = serialDevice.ReadLine();
      Debug.WriteLine($"ReadLine after:{line}");

            // check for +EVT:JOINED
      if (line.StartsWith("+EVT:JOINED"))
      {
            OnJoinCompletion?.Invoke(true);

            continue;
      }

      if (line.StartsWith("+EVT:JOIN FAILED"))
      {
	     OnJoinCompletion?.Invoke(false);

         continue;
      }

      if (line.StartsWith("+EVT:SEND CONFIRMED OK"))
      {
         OnMessageConfirmation?.Invoke();

         continue;
      }

      // Check for A/B/C downlink message
      if (line.StartsWith("+EVT:RX_1") || line.StartsWith("+EVT:RX_2") || line.StartsWith("+EVT:RX_3") || line.StartsWith("+EVT:RX_C"))
      {
         string[] fields1 = line.Split(' ', ',');

         int rssi = int.Parse(fields1[3]);
         int snr = int.Parse(fields1[6]);
 
         line = serialDevice.ReadLine();
         Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} UNICAST :{line}");

         line = serialDevice.ReadLine();
         Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Payload:{line}");

         string[] fields2 = line.Split(':');

         int port = int.Parse(fields2[1]);
         string payload = fields2[2];

         OnReceiveMessage?.Invoke(port, rssi, snr, payload);

         continue;
      }

      try
      {
         this.serialDevice.ReadTimeout = 3000;

         Debug.WriteLine("ReadLine Result");
         line = serialDevice.ReadLine();
         Debug.WriteLine($"ReadLine Result after:{line}");

         switch (line)
         {
            case "OK":
               result = Result.Success;
               break;
         case "AT_ERROR":
               result = Result.Error;
               break;
         case "AT_PARAM_ERROR":
               result = Result.ParameterError;
               break;
         case "AT_BUSY_ERROR":
               result = Result.BusyError;
               break;
         case "AT_TEST_PARAM_OVERFLOW":
               result = Result.ParameterOverflow;
               break;
         case "AT_NO_NETWORK_JOINED":
               result = Result.NotJoined;
               break;
         case "AT_RX_ERROR":
               result = Result.ReceiveError;
               break;
         case "AT_DUTYCYLE_RESTRICTED":
               result = Result.DutyCycleRestricted;
               break;
         default:
               result = Result.Undefined;
               break;
         }
      }
      catch (TimeoutException) 
      {
         result = Result.Timeout;
      }
   atExpectedEvent.Set();
}

The code is not suitable for production but it confirmed my thread based approach works. I need to add code to shutdown the message processing thread in a controlled way, support for Class B & C devices, replace the OnJoinCompletionHandler timer magic numbers and soak test for 5-7 days.

Visual Studio Displaying RAK3172 device joining network then sending messages

In the Visual Studio 2019 debug output I could see messages getting sent and then after a short delay they were visible in the TTN console.

TTN Displaying RAK3172 device joining network then sending messages

.NET Core RAK3172 LoRaWAN library Part3

Nasty ABP connect

After getting basic connectivity for my RAK3172 test rig sorted I wanted to see if I could get the device connected to The Things Network(TTN) via the RAK7246G LPWAN Developer Gateway on my bookcase.

Raspberry Pi3 with Grove Base Hat and RAK3172 Breakout (using UART2)

My Activation By Personalisation (ABP) implementation is very “nasty” (just like the OTAA one) I have assumed that there would be no timeouts or failures and I only send one BCD message “48656c6c6f204c6f526157414e” which is “hello LoRaWAN”.

The code just sequentially steps through the necessary configuration to join the TTN network with a suitable delay after each command is sent.

//---------------------------------------------------------------------------------
// Copyright (c) September 2021, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.NetCore.RAK3172.NetworkJoinABP
{
	using System;
	using System.Diagnostics;
	using System.IO.Ports;
	using System.Threading;

	public class Program
	{
		private const string SerialPortId = "/dev/ttyS0";
		private const string DevAddress = "...";
		private const string NwksKey = "...";
		private const string AppsKey = "...";
		private const byte MessagePort = 1;
		private const string Payload = "A0EEE456D02AFF4AB8BAFD58101D2A2A"; // Hello LoRaWAN

		public static void Main()
		{
			string response;

			Debug.WriteLine("devMobile.IoT.NetCore.Rak3172.NetworkJoinOTAA starting");

			Debug.WriteLine(String.Join(",", SerialPort.GetPortNames()));

			try
			{
				using (SerialPort serialPort = new SerialPort(SerialPortId))
				{
					// set parameters
					serialPort.BaudRate = 9600;
					serialPort.DataBits = 8;
					serialPort.Parity = Parity.None;
					serialPort.StopBits = StopBits.One;
					serialPort.Handshake = Handshake.None;

					serialPort.ReadTimeout = 5000;

					serialPort.NewLine = "\r\n";

					serialPort.Open();

					// clear out the RX buffer
					response = serialPort.ReadExisting();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");
					Thread.Sleep(500);

					// Set the Working mode to LoRaWAN
					Console.WriteLine("Set Work mode");
					serialPort.WriteLine("AT+NWM=1");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the Region to AS923
					Console.WriteLine("Set Region");
					serialPort.WriteLine("AT+BAND=8-1");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the JoinMode
					Console.WriteLine("Set Join mode");
					serialPort.WriteLine("AT+NJM=0");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the device address
					Console.WriteLine("Set Device Address");
					serialPort.WriteLine($"AT+DEVADDR={DevAddress}");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the network session key
					Console.WriteLine("Set Network Session Key");
					serialPort.WriteLine($"AT+NWKSKEY={NwksKey}");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the application session key
					Console.WriteLine("Set application Session Key");
					serialPort.WriteLine($"AT+APPSKEY={AppsKey}");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the Confirm flag
					Console.WriteLine("Set Confirm off");
					serialPort.WriteLine("AT+CFM=0");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Join the network
					Console.WriteLine("Start Join");
					serialPort.WriteLine("AT+JOIN=1:0:10:2");

					// Read the blank line
					response = serialPort.ReadLine();

					// Read the Result
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					Thread.Sleep(10000);

					// Read the +EVT:JOINED
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					while (true)
					{
						Console.WriteLine("Sending");
						serialPort.WriteLine($"AT+SEND={MessagePort}:{Payload}");

						// Read the blank line
						response = serialPort.ReadLine();

						// Read the result
						Console.WriteLine("Send result");
						response = serialPort.ReadLine();
						Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

						Thread.Sleep(300000);
					}
				}
			}
			catch (Exception ex)
			{
				Debug.WriteLine(ex.Message);
			}
		}
	}
}

The code is not suitable for production but it confirmed my software and hardware configuration worked.

In the Visual Studio 2019 debug output I could see messages getting sent and then after a short delay they were visible in the TTN console.

The RAK3172 command format is quite different from other modules I have used e.g. Requesting the firmware version information

  • TX- AT+VER=?
  • RX- Blank Line
  • RX- V1.0.2
  • RX- OK

Requesting the APPEUI

  • TX- AT+DEVADDR=?
  • RX- 11223344
  • RX- Blank line
  • RX- OK

I think the RAK3172 module ships with a default DEVEUI so in this code and my library I have assumed it will be configured as part of a “provisioning” process.

.NET Core RAK3172 LoRaWAN library Part2

Nasty OTAA connect

After getting basic connectivity for my RAK3172 test rig sorted I wanted to see if I could get the device connected to The Things Network(TTN) via the RAK7246G LPWAN Developer Gateway on my bookcase.

Raspberry Pi3 with Grove Base Hat and RAK3172 Breakout (using UART2)

My Over the Air Activation (OTAA) implementation is very “nasty” I have assumed that there would be no timeouts or failures and I only send one BCD message “48656c6c6f204c6f526157414e” which is “hello LoRaWAN”.

The code just sequentially steps through the necessary configuration to join the TTN network with a suitable delay after each command is sent.

//---------------------------------------------------------------------------------
// Copyright (c) September 2021, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.NetCore.RAK3172.NetworkJoinOTAA
{
	using System;
	using System.Diagnostics;
	using System.IO.Ports;
	using System.Threading;

	public class Program
	{
		private const string SerialPortId = "/dev/ttyS0";
		private const string AppEui = "...";
		private const string AppKey = "...";
		private const byte MessagePort = 1;
		private const string Payload = "A0EEE456D02AFF4AB8BAFD58101D2A2A"; // Hello LoRaWAN

		public static void Main()
		{
			string response;

			Debug.WriteLine("devMobile.IoT.NetCore.Rak3172.NetworkJoinOTAA starting");

			Debug.WriteLine(String.Join(",", SerialPort.GetPortNames()));

			try
			{
				using (SerialPort serialPort = new SerialPort(SerialPortId))
				{
					// set parameters
					serialPort.BaudRate = 9600;
					serialPort.DataBits = 8;
					serialPort.Parity = Parity.None;
					serialPort.StopBits = StopBits.One;
					serialPort.Handshake = Handshake.None;

					serialPort.ReadTimeout = 5000;

					serialPort.NewLine = "\r\n";

					serialPort.Open();

					// clear out the RX buffer
					response = serialPort.ReadExisting();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");
					Thread.Sleep(500);


					// Set the Working mode to LoRaWAN
					Console.WriteLine("Set Work mode");
					serialPort.WriteLine("AT+NWM=1");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the Region to AS923
					Console.WriteLine("Set Region");
					serialPort.WriteLine("AT+BAND=8-1");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the JoinMode
					Console.WriteLine("Set Join mode");
					serialPort.WriteLine("AT+NJM=1");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the appEUI
					Console.WriteLine("Set App Eui");
					serialPort.WriteLine($"AT+APPEUI={AppEui}");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the appKey
					Console.WriteLine("Set App Key");
					serialPort.WriteLine($"AT+APPKEY={AppKey}");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the Confirm flag
					Console.WriteLine("Set Confirm off");
					serialPort.WriteLine("AT+CFM=0");
					// Read the blank line
					response = serialPort.ReadLine();
					// Read the response
					response = serialPort.ReadLine(); 
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Join the network
					Console.WriteLine("Start Join");
					serialPort.WriteLine("AT+JOIN=1:0:10:2");

					// Read the blank line
					response = serialPort.ReadLine();

					// Read the Result
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					Thread.Sleep(10000);

					// Read the +EVT:JOINED
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					while (true)
					{
						Console.WriteLine("Sending");
						serialPort.WriteLine($"AT+SEND={MessagePort}:{Payload}");

						// Read the blank line
						response = serialPort.ReadLine();

						// Read the result
						Console.WriteLine("Send result");
						response = serialPort.ReadLine();
						Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

						Thread.Sleep(300000);
					}
				}
			}
			catch (Exception ex)
			{
				Debug.WriteLine(ex.Message);
			}
		}
	}
}

The code is not suitable for production but it confirmed my software and hardware configuration worked.

In the Visual Studio 2019 debug output I could see messages getting sent and then after a short delay they were visible in the TTN console.

The RAK3172 command format is quite different from other modules I have used e.g. Requesting the firmware version information

  • TX- AT+VER=?
  • RX- Blank Line
  • RX- V1.0.2
  • RX- OK

Requesting the APPEUI

  • TX- AT+APPEUI=?
  • RX- 1122334455667788
  • RX- Blank line
  • RX- OK

I think the RAK3172 module ships with a default DEVEUI so in this code and my library I have assumed it will be configured as part of a “provisioning” process.

.NET Core Seeed LoRaE5 LoRaWAN library Part2

Nasty OTAA connect

After getting basic connectivity for my Seeed LoRa-E5 test rig sorted I used RAK7246G LPWAN Developer Gateway on my bookcase to connect to The Things Network(TTN)

Seeed LoRa-E5 Development kit connected to Gove bas shield on a Raspberry PI3

My Over the Air Activation (OTAA) implementation is very “nasty” I have assumed that there would be no timeouts or failures and I only send one BCD message “48656c6c6f204c6f526157414e” which is “hello LoRaWAN”.

The code just sequentially steps through the necessary configuration to join the TTN network with a suitable delay after each command is sent. There also appeared to be quite a variation in response times, especially for joining the network(most probably network related) and the progress of sending a message.

//---------------------------------------------------------------------------------
// Copyright (c) September 2021, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.NetCore.SeeedLoRaE5.NetworkJoinOTAA
{
	using System;
	using System.Diagnostics;
	using System.IO.Ports;
	using System.Threading;

	class Program
	{
		private const string SerialPortId = "/dev/ttyS0";

		private const string AppKey = "................................";
		private const string AppEui = "................";

		private const byte MessagePort = 15;

		//private const string Payload = "48656c6c6f204c6f526157414e"; // Hello LoRaWAN
		private const string Payload = "01020304"; // AQIDBA==
		//private const string Payload = "04030201"; // BAMCAQ==

		public static void Main()
		{
			string response;

			Debug.WriteLine("devMobile.IoT.SeeedLoRaE5.NetworkJoinOTAA starting");

			Debug.WriteLine(String.Join(",", SerialPort.GetPortNames()));

			try
			{
				using (SerialPort serialDevice = new SerialPort(SerialPortId))
				{
					// set parameters
					serialDevice.BaudRate = 9600;
					serialDevice.Parity = Parity.None;
					serialDevice.StopBits = StopBits.One;
					serialDevice.Handshake = Handshake.None;
					serialDevice.DataBits = 8;

					serialDevice.ReadTimeout = 10000;

					serialDevice.NewLine = "\r\n";

					serialDevice.Open();

					// clear out the RX buffer
					serialDevice.ReadExisting();
					response = serialDevice.ReadExisting();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");
					Thread.Sleep(500);

					// Set the Region to AS923
					serialDevice.WriteLine("AT+DR=AS923\r\n");
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					// Set the Join mode
					serialDevice.WriteLine("AT +MODE=LWOTAA\r\n");
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					// Set the appEUI
					serialDevice.WriteLine($"AT+ID=AppEui,\"{AppEui}\"\r\n");
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					// Set the appKey
					serialDevice.WriteLine($"AT+KEY=APPKEY,{AppKey}\r\n");
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					// Set the port number
					serialDevice.WriteLine($"AT+PORT={MessagePort}\r\n");
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					// Join the network
					serialDevice.WriteLine("AT+JOIN\r\n");

					// Join start
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					// JOIN normal
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					Thread.Sleep(5000);

					// network joined
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					// Net ID
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					// Join done
					response = serialDevice.ReadLine();
					Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

					while (true)
					{
						Debug.WriteLine("Sending");

						serialDevice.WriteLine($"AT+MSGHEX=\"{Payload}\"\r\n");

						// Start
						response = serialDevice.ReadLine();
						Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

						// Fpending
						response = serialDevice.ReadLine();
						Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

						//Read metrics
						response = serialDevice.ReadLine();
						Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

						//Done
						response = serialDevice.ReadLine();
						Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");

						Thread.Sleep(30000);
					}
				}
			}
			catch (Exception ex)
			{
				Debug.WriteLine(ex.Message);
			}
		}
	}
}

The code is not suitable for production but it confirmed my software and hardware configuration worked.

Visual Studio debugger output window showing network join and sensing a message

In the Visual Studio 2019 debug output I could see messages getting sent and then after a short delay they were visible in the TTN console.

The Things Industries Live Data view showing network join and sensing a message

Most of the LoRaWAN modems I have worked with reply “OK” when a command is successful. The SeeedLoRa-E5 often returns the payload of the request in the response which makes the code a little bit more complex.

AppEui command structure in AT Command documentation

For example the AppEui can be passed in as “00:00:00:00:00:00:00:00” or “0000000000000000” but in the response the format is always “00:00:00:00:00:00:00:00”

.NET Core RAK811 LoRaWAN library Part2

Nasty OTAA connect

After getting basic connectivity for my IoT LoRa Node pHAT for Raspberry Pi test rig sorted I wanted to see if I could get the device connected to The Things Network(TTN) via the RAK7246G LPWAN Developer Gateway on my bookcase.

IoT LoRa Node pHAT for Raspberry Pi mounted on a Raspberry PI3

My Over the Air Activation (OTAA) implementation is very “nasty” I have assumed that there would be no timeouts or failures and I only send one BCD message “48656c6c6f204c6f526157414e” which is “hello LoRaWAN”.

The code just sequentially steps through the necessary configuration to join the TTN network with a suitable delay after each command is sent. I had some problems with re-opening the serial port if my application had previously failed with an uncaught exception. After some experimentation I added some code to ensure the port was closed when an exception occurred.

//---------------------------------------------------------------------------------
// Copyright (c) September 2021, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.NetCore.Rak811.NetworkJoinOTAA
{
	using System;
	using System.Diagnostics;
	using System.IO.Ports;
	using System.Threading;

	public class Program
	{
		private const string SerialPortId = "/dev/ttyS0";
		private const string AppEui = "...";
		private const string AppKey = "...";
		private const byte MessagePort = 1;
		private const string Payload = "A0EEE456D02AFF4AB8BAFD58101D2A2A"; // Hello LoRaWAN

		public static void Main()
		{
			string response;

			Debug.WriteLine("devMobile.IoT.NetCore.Rak811.NetworkJoinOTAA starting");

			Debug.WriteLine(String.Join(",", SerialPort.GetPortNames()));

			try
			{
				using (SerialPort serialPort = new SerialPort(SerialPortId))
				{
					// set parameters
					serialPort.BaudRate = 9600;
					serialPort.DataBits = 8;
					serialPort.Parity = Parity.None;
					serialPort.StopBits = StopBits.One;
					serialPort.Handshake = Handshake.None;

					serialPort.ReadTimeout = 5000;

					serialPort.NewLine = "\r\n";

					serialPort.Open();

					// clear out the RX buffer
					response = serialPort.ReadExisting();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");
					Thread.Sleep(500);

					// Set the Working mode to LoRaWAN
					Console.WriteLine("Set Work mode");
					serialPort.WriteLine("at+set_config=lora:work_mode:0");
					Thread.Sleep(5000);
					response = serialPort.ReadExisting();
					response = response.Trim('\0');
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the Region to AS923
					Console.WriteLine("Set Region");
					serialPort.WriteLine("at+set_config=lora:region:AS923");
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the JoinMode
					Console.WriteLine("Set Join mode");
					serialPort.WriteLine("at+set_config=lora:join_mode:0");
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the appEUI
					Console.WriteLine("Set App Eui");
					serialPort.WriteLine($"at+set_config=lora:app_eui:{AppEui}");
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the appKey
					Console.WriteLine("Set App Key");
					serialPort.WriteLine($"at+set_config=lora:app_key:{AppKey}");
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					// Set the Confirm flag
					Console.WriteLine("Set Confirm off");
					serialPort.WriteLine("at+set_config=lora:confirm:0");
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");
					
					// Join the network
					Console.WriteLine("Start Join");
					serialPort.WriteLine("at+join");
					Thread.Sleep(10000);
					response = serialPort.ReadLine();
					Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

					while (true)
					{
						Console.WriteLine("Sending");
						serialPort.WriteLine($"at+send=lora:{MessagePort}:{Payload}");
						Thread.Sleep(1000);

						// The OK
						Console.WriteLine("Send result");
						response = serialPort.ReadLine();
						Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

						// The Signal strength information etc.
						Console.WriteLine("Network confirmation");
						response = serialPort.ReadLine();
						Debug.WriteLine($"RX :{response.Trim()} bytes:{response.Length}");

						Thread.Sleep(20000);
					}
				}
			}
			catch (Exception ex)
			{
				Debug.WriteLine(ex.Message);
			}
		}
	}
}

The code is not suitable for production but it confirmed my software and hardware configuration worked.

In the Visual Studio 2019 debug output I could see messages getting sent and then after a short delay they were visible in the TTN console.

The Things Industries Live Data Tab showing my device connecting and sending messages

The RAK811 doesn’t fully implement the AS923 specification but I have a work around detailed in another post.