.NET nanoFramework RAK3172 LoRaWAN library OTAA Join

When I first tried Over The Air Activation(OTAA) to connect to The Things Network(TTN) with my RAKwireless RAK3172 module it didn’t work. I had flashed my Sparkfun Thing plus ESP32 WROOM with the ESP32 PSRAM binaries (Thanks AlbertK on the nanoFramework Discord).

nanoff --target ESP32_REV0 --serialport COM17 --update
RAK3172 Sparkfun ESP32 WROOM Testrig

I modified the NetworkJoinOTAA sample (based on the asynchronous version of BreakOutSerial) to send the sequence of AT commands to join The Things Network(TTN) and display the responses.

#define ST_STM32F769I_DISCOVERY      // nanoff --target ST_STM32F769I_DISCOVERY --update 
//#define ESP32_WROOM   //nanoff --target ESP32_REV0 --serialport COM17 --update

namespace devMobile.IoT.LoRaWAN.nanoFramework.RAK3172
{
	using System;
	using System.Diagnostics;
	using System.IO.Ports;
	using System.Threading;
#if ESP32_WROOM
	using global::nanoFramework.Hardware.Esp32; //need NuGet nanoFramework.Hardware.Esp32
#endif

	public class Program
	{
#if ESP32_WROOM
		private const string SerialPortId = "COM2";
#endif
#if ST_STM32F769I_DISCOVERY
		private const string SerialPortId = "COM6";
#endif
		private const string DevEui = "...";
		private const string AppEui = "...";
		private const string AppKey = "...";
		private const byte MessagePort = 1;
		private const string Payload = "A0EEE456D02AFF4AB8BAFD58101D2A2A"; // Hello LoRaWAN

		public static void Main()
		{
			Debug.WriteLine("devMobile.IoT.LoRaWAN.nanoFramework.RAK3172 NetworkJoinOTAA starting");

			try
			{
				// set GPIO functions for COM2 (this is UART1 on ESP32)
#if ESP32_WROOM
				Configuration.SetPinFunction(Gpio.IO17, DeviceFunction.COM2_TX);
				Configuration.SetPinFunction(Gpio.IO16, DeviceFunction.COM2_RX);
#endif

				Debug.Write("Ports:");
				foreach (string port in SerialPort.GetPortNames())
				{
					Debug.Write($" {port}");
				}
				Debug.WriteLine("");

				using (SerialPort serialPort = new SerialPort(SerialPortId))
				{
					// set parameters
					serialPort.BaudRate = 115200;
					serialPort.Parity = Parity.None;
					serialPort.DataBits = 8;
					serialPort.StopBits = StopBits.One;
					serialPort.Handshake = Handshake.None;
					serialPort.NewLine = "\r\n";
					serialPort.ReadTimeout = 1000;

					serialPort.DataReceived += SerialDevice_DataReceived;

					serialPort.Open();

					serialPort.WatchChar = '\n';

					serialPort.ReadExisting(); // Running at 115K2 this was necessary

					// Set the devEUI
					Console.WriteLine("Set Dev Eui");
					serialPort.WriteLine($"AT+DEVEUI={DevEui}");

					// Set the Working mode to LoRaWAN
					Console.WriteLine("Set Work mode");
					serialPort.WriteLine("AT+NWM=1");

					// Set the Region to AS923
					Console.WriteLine("Set Region");
					serialPort.WriteLine("AT+BAND=8");

					// Set the JoinMode
					Console.WriteLine("Set Join mode");
					serialPort.WriteLine("AT+NJM=1");

					// Set the appEUI
					Console.WriteLine("Set App Eui");
					serialPort.WriteLine($"AT+APPEUI={AppEui}");

					// Set the appKey
					Console.WriteLine("Set App Key");
					serialPort.WriteLine($"AT+APPKEY={AppKey}");

					// Set the Confirm flag
					Console.WriteLine("Set Confirm off");
					serialPort.WriteLine("AT+CFM=0");

					// Join the network
					Console.WriteLine("Start Join");
					serialPort.WriteLine("AT+JOIN=1:0:10:2");

					// Wait for the +EVT:JOINED

					while (true)
					{
						Console.WriteLine("Sending");
						serialPort.WriteLine($"AT+SEND={MessagePort}:{Payload}");

						Thread.Sleep(300000);
					}
				}
			}
			catch (Exception ex)
			{
				Debug.WriteLine(ex.Message);
			}
		}

		private static void SerialDevice_DataReceived(object sender, SerialDataReceivedEventArgs e)
		{
			SerialPort serialPort = (SerialPort)sender;

			switch (e.EventType)
			{
				case SerialData.Chars:
					break;
 
				case SerialData.WatchChar:
					string response = serialPort.ReadExisting();
					Debug.Write(response);
					break;
				default:
					Debug.Assert(false, $"e.EventType {e.EventType} unknown");
					break;
			}
		}
	}
}
Visual Studio Debug Output displaying Configuration-Join-Send
Things Network(TTN) Live Data tab display successful join then D2C message

The NetworkJoinOTAA application assumes that all of the AT commands succeed. I usually step though the application in the debugger so I can monitor progress.

.NET Core web API + Dapper – Readonly query workloads with Geo Replication

Geo Replication process

One of the easiest ways to create read-only replicas of an Azure SQL Database database is with Active geo-replication(it’s also useful for disaster recovery with geo-failure to a geo-secondary in a different Azure Region).

The first step was to remove all the Microsoft SQL Server features used in the The World Wide Importers database (e.g. Memory Optimized tables) which were not supported by the Azure SQL Database tier I was using.

SQL Service Management Studio(SSMS) Deploy to Azure wizard

I then used the “Deploy Database Wizard” to copy my modified World Wide Importers database to an Azure SQL Database.

Azure Portal Primary Database in SQL Service Management Studio(SSMS)

I then created replicas in the same region (if the application had a global customer base creating read only geo replicas in regions close to users might be worth considering) for the read-only queries.

Azure SQL Database no replicas configured
Azure Portal Create Geo Replica
Azure Portal Create Geo Replica Confirmation

I created four replicas which is the maximum number supported. If more replicas were required a secondary of a secondary (a process known as chaining) could be use to create additional geo-replicas

Azure Portal Primary Database and four Geo-replicas
Azure Portal Primary Database and Geo-replicas in SQL Service Management Studio(SSMS)
Azure Application Insights showing multiple Geo-Replicas being used.

The Azure Database Geo-replication was pretty easy to setup. For a production scenario where only a portion of the database (e.g. shaped by Customer or Geography) is required it might not be the “right hammer”.

WebAPI Dapper Azure Resource Group

The other limitation I encountered was the resources used by the replication of “transaction processing” tables (in the World Wide Importers database tables like the Sales.OrderLines, Sales.CustomerTransactions etc.) which often wouldn’t be required for read-only applications.

.NET Core web API + Dapper – Offloading readonly query workloads.

A Scaling Out Scenario

Initially this was going to be a post about Sharding but after some discussion with my manager at one of the companies I work for it evolved into a post about using the Dapper Object Relational Mapper(ORM) with Azure SQL Database and Active geo-replication or SQL Data Sync for Azure to offload read-only query workloads to improve scalability.

The company builds a Software as a Service(Saas) product for managing portfolios of foreign currency forwardsoptionsswaps etc. Part of the solution has a React application which customers use to get an “aggregated” view of their exposure to different currency pairs for different periods etc.

The database queries to lookup reference data (forward curves etc.), return a shaped dataset for each supported instrument type, then “aggregating” the information with C# code consumes significant database and processing resources.

The first step was to remove all the Microsoft SQL Server features used in the The World Wide Importers database (e.g. Memory Optimized tables) which were not supported by the Azure SQL Database vCore or DTU tier I had was using.

I then uploaded my modified World Wide Importers database to an Azure SQL Database Server.

SQL Server Management studio with World Wide Importers open in Object Explorer tab
Azure SQL Database SQL Server Management Studio (SSMS) initial database

I then created read only replicas of the original database to use for scaling out in my demo application.

Azure SQL Database SQL Server Management Studio (SSMS) with replicas databases

The configuration strings of the read-only replicas are loaded as the application starts.

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
	services.AddControllers();

	var errorHandlerSettings = Configuration.GetSection(nameof(ErrorHandlerSettings));
	services.Configure<ErrorHandlerSettings>(errorHandlerSettings);

	var readonlyReplicaServersConnectionStringSettings = Configuration.GetSection("ReadonlyReplicaServersConnectionStringSettings");
	services.Configure<List<string>>(readonlyReplicaServersConnectionStringSettings);

	services.AddResponseCaching();

	services.AddDapperForMSSQL();
#if DAPPER_EXTENSIONS_CACHE_MEMORY
	services.AddDapperCachingInMemory(new MemoryConfiguration
	{
		AllMethodsEnableCache = false
 	});
#endif
#if DAPPER_EXTENSIONS_CACHE_REDIS
	services.AddDapperCachingInRedis(new RedisConfiguration
	{
		AllMethodsEnableCache = false,
		KeyPrefix = Configuration.GetValue<string>("RedisKeyPrefix"),
		ConnectionString = Configuration.GetConnectionString("RedisConnection")
	}); 
#endif
	services.AddApplicationInsightsTelemetry();
}

Then code was added to the controller to randomly select which read-only replica to use. More complex approaches were considered but not implemented for the initial version.

[ApiController]
[Route("api/[controller]")]
public class StockItemsReadonlyReplicasController : ControllerBase
{
    private readonly ILogger<StockItemsReadonlyReplicasController> logger;
    private readonly List<string> readonlyReplicasConnectionStrings;

    public StockItemsReadonlyReplicasController(ILogger<StockItemsReadonlyReplicasController> logger, IOptions<List<string>> readonlyReplicasServerConnectionStrings)
    {
        this.logger = logger;

        this.readonlyReplicasConnectionStrings = readonlyReplicasServerConnectionStrings.Value;
    }

    [HttpGet]
    public async Task<ActionResult<IAsyncEnumerable<Model.StockItemListDtoV1>>> Get()
    {
        IEnumerable<Model.StockItemListDtoV1> response = null;

        if (readonlyReplicasConnectionStrings.Count == 0)
        {
            logger.LogError("No readonly replica server Connection strings configured");

            return this.StatusCode(StatusCodes.Status500InternalServerError);
        }

        Random random = new Random(); // maybe this should be instantiated ever call, but "danger here by thy threading"

        string connectionString = readonlyReplicasConnectionStrings[random.Next(0, readonlyReplicasConnectionStrings.Count)];

        logger.LogTrace("Connection string {connectionString}", connectionString);

        using (SqlConnection db = new SqlConnection(connectionString))
        {
            response = await db.QueryAsync<Model.StockItemListDtoV1>(sql: @"SELECT [StockItemID] as ""ID"", [StockItemName] as ""Name"", [RecommendedRetailPrice], [TaxRate] FROM [Warehouse].[StockItems]", commandType: CommandType.Text);
        }

        return this.Ok(response);
    }
}

The Read-only replica server connection string setup template in appsettings.Development.json.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "ConnectionStrings": {
    "WorldWideImportersDatabase": "ThisIsNotTheDatabaseConnectionStringYouAreLookingFor",
    "DefaultConnection": "ThisIsNotTheDefaultDatabaseConnectionStringYouAreLookingFor",
    "RedisConnection": "ThisIsNotTheRedisConnectionStringYouAreLookingFor"
  },
  "ReadonlyReplicaServersConnectionStringSettings": [
    "ThisIsNotTheReadonlyReplicaDatabaseConnectionStringYouAreLookingFor",
    "ThisIsNotTheReadonlyReplicaDatabaseConnectionStringYouAreLookingFor",
    "ThisIsNotTheReadonlyReplicaDatabaseConnectionStringYouAreLookingFor",
    "ThisIsNotTheReadonlyReplicaDatabaseConnectionStringYouAreLookingFor"
  ],
  "ApplicationInsights": {
    "ConnectionString": "ThisIsNotTheApplicationInsightsConnectionStringYouAreLookingFor"
  },
  "ErrorHandlerSettings": {
    "Detail": "Default detail",
    "Title": "Default title",
    "UrlSpecificSettings": {
      "localhost": {
        "Title": "Title for localhost",
        "Detail": "Detail for localhost"
      },
      "127.0.0.1": {
        "Title": "Title for 127.0.0.1",
        "Detail": "Detail for 127.0.0.1"
      }
    }
  }
}

The Manage UserSecrets(Secrets.json) functionality was used for testing on my development machine. In production Azure App Service the array of connections strings was configured with ReadonlyReplicaServersConnectionStringSettings:0, ReadonlyReplicaServersConnectionStringSettings:1 etc. syntax

Sample application Azure App Service Configuration
Azure Application Insights with connections to different read-only replicas highlighted

I had incorrectly configured the firewall on one of the read-only replica database servers so roughly one in four connection attempts failed.

Azure Application Insights failed database connection displayed

The customer’s application was also fairly compute intensive so we configure the Azure App Service to Auto scale based on the CPU load.,

Azure app service horizontal auto scale configuration

I’ll outline the configuration of Active geo-replication or SQL Data Sync for Azure in a couple of future posts.

The current solution works but I need to tidy up few issues like the StockItemsReadonlyReplicasController getting constructor getting a bit “chunky”.

.NET nanoFramework RAK3172 LoRaWAN library basic connectivity

I have been working on a .NET nanoFramework library for the RAKwireless RAK3172 module for the last couple of weeks. The devices had been in a box under my desk for a couple of months so first step was to flash them with the latest firmware using my FTDI test harness.

RAK 3172 STM32F769I Discovery test rig

I use two hardware configurations for testing

My sample code has compile time options for synchronous and asynchronous operation. I also include the different nanoff command lines to make updating the test devices easier.

//---------------------------------------------------------------------------------
// Copyright (c) May 2022, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// https://docs.rakwireless.com/Product-Categories/WisDuo/RAK4200-Breakout-Board/AT-Command-Manual/
//---------------------------------------------------------------------------------
#define SERIAL_ASYNC_READ
//#define SERIAL_THREADED_READ
#define ST_STM32F769I_DISCOVERY      // nanoff --target ST_STM32F769I_DISCOVERY --update 
//#define ESP32_WROOM   // nanoff --target ESP32_REV0 --serialport COM17 --update
...

namespace devMobile.IoT.LoRaWAN.nanoFramework.RAK3172
{
	using System;
	using System.Diagnostics;
	using System.IO.Ports;
	using System.Threading;
#if ESP32_WROOM
	using global::nanoFramework.Hardware.Esp32; //need NuGet nanoFramework.Hardware.Esp32
#endif

	public class Program
	{
		private static SerialPort _SerialPort;
#if SERIAL_THREADED_READ
		private static Boolean _Continue = true;
#endif
#if ESP32_WROOM
		private const string SerialPortId = "COM2";
#endif
...
#if ST_STM32F769I_DISCOVERY
		private const string SerialPortId = "COM6";
#endif

		public static void Main()
		{
#if SERIAL_THREADED_READ
			Thread readThread = new Thread(SerialPortProcessor);
#endif

			Debug.WriteLine("devMobile.IoT.LoRaWAN.nanoFramework.RAK3172 BreakoutSerial starting");

			try
			{
				// set GPIO functions for COM2 (this is UART1 on ESP32)
#if ESP32_WROOM
				Configuration.SetPinFunction(Gpio.IO16, DeviceFunction.COM2_TX);
				Configuration.SetPinFunction(Gpio.IO17, DeviceFunction.COM2_RX);
#endif

				Debug.Write("Ports:");
				foreach (string port in SerialPort.GetPortNames())
				{
					Debug.Write($" {port}");
				}
				Debug.WriteLine("");

				using (_SerialPort = new SerialPort(SerialPortId))
				{
					// set parameters
					_SerialPort.BaudRate = 115200;
					_SerialPort.Parity = Parity.None;
					_SerialPort.DataBits = 8;
					_SerialPort.StopBits = StopBits.One;
					_SerialPort.Handshake = Handshake.None;
					_SerialPort.NewLine = "\r\n";
					_SerialPort.ReadTimeout = 1000;

					//_SerialPort.WatchChar = '\n'; // May 2022 WatchChar event didn't fire github issue https://github.com/nanoframework/Home/issues/1035

#if SERIAL_ASYNC_READ
					_SerialPort.DataReceived += SerialDevice_DataReceived;
#endif

					_SerialPort.Open();

					_SerialPort.WatchChar = '\n';

#if SERIAL_THREADED_READ
					readThread.Start();
#endif

					for (int i = 0; i < 5; i++)
					{
						string atCommand;
						atCommand = "AT+VER=?";

                  Debug.WriteLine("");
						Debug.WriteLine($"{i} TX:{atCommand} bytes:{atCommand.Length}--------------------------------");
						_SerialPort.WriteLine(atCommand);

						Thread.Sleep(5000);
					}
				}
				Debug.WriteLine("Done");
			}
			catch (Exception ex)
			{
				Debug.WriteLine(ex.Message);
			}
		}

#if SERIAL_ASYNC_READ
		private static void SerialDevice_DataReceived(object sender, SerialDataReceivedEventArgs e)
		{
			SerialPort serialPort = (SerialPort)sender;

			switch (e.EventType)
			{
				case SerialData.Chars:
					break;

				case SerialData.WatchChar:
					string response = serialPort.ReadExisting();
					Debug.Write(response);
					break;
				default:
					Debug.Assert(false, $"e.EventType {e.EventType} unknown");
					break;
			}
		}
#endif

#if SERIAL_THREADED_READ
		public static void SerialPortProcessor()
		{

			while (_Continue)
			{
				try
				{
					string response = _SerialPort.ReadLine();
					//string response = _SerialPort.ReadExisting();
					Debug.Write(response);
				}
				catch (TimeoutException ex) 
				{
					Debug.WriteLine($"Timeout:{ex.Message}");
				}
			}
		}
#endif
	}
}

When I requested the RAK3172 version information with “AT+VER=?” the response was spilt over two lines which is a bit of a Pain in the Arse (PitA). The RAK3172 firmware also defaults 115200 baud which seems overkill considering the throughput of a LoRaWAN link.

Visual Studio Debug Output of Breakout Serial Application

While building the test application I encountered a few issues (STM32F769I DISCOVERY SerialPort.GetPortNames() port name text gets shorter, STM32F769I DISCOVERY Inconsistent SerialPort WatchChar behaviour after erase->power cycle->run & erase->run and No SerialPort.WatchChar events if WatchChar set before SerialPort opened) which slowed down development. The speed the nanoFramework team triages then fixes issues is amazing for a team of volunteers dotted around the world.

.NET Core web API + Dapper – Error handling

Failure is an Option

For some historical reason I can’t remember my controllers often had an outer try/catch and associated logging. I think may have been ensure no “sensitive” information was returned to the caller even if the application was incorrectly deployed. So I could revisit my approach I added a controller with two methods one which returns an HTTP 500 error and another which has un-caught exception.

[Route("api/[controller]")]
[ApiController]
public class StockItemsNok500Controller : ControllerBase
{
	private readonly string connectionString;
	private readonly ILogger<StockItemsNok500Controller> logger;

	public StockItemsNok500Controller(IConfiguration configuration, ILogger<StockItemsNok500Controller> logger)
	{
		this.connectionString = configuration.GetConnectionString("WorldWideImportersDatabase");

		this.logger = logger;
	}

	public async Task<ActionResult<IAsyncEnumerable<Model.StockItemListDtoV1>>> Get500()
	{
		IEnumerable<Model.StockItemListDtoV1> response = null;

		try
		{
			using (SqlConnection db = new SqlConnection(this.connectionString))
			{
				response = await db.QueryWithRetryAsync<Model.StockItemListDtoV1>(sql: @"SELECT [StockItemID] as ""ID"", [StockItemName] as ""Name"", [RecommendedRetailPrice], [TaxRate] FROM [Warehouse].[StockItem500]", commandType: CommandType.Text);
			}
		}
		catch (SqlException ex)
		{
			logger.LogError(ex, "Retrieving list of StockItems");

			return this.StatusCode(StatusCodes.Status500InternalServerError);
		}

		return this.Ok(response);
	}
}

The information returned to a caller was generic and the only useful information was the “traceId”.

StockItemsNok500Controller error page
[Route("api/[controller]")]
[ApiController]
public class StockItemsNokExceptionController : ControllerBase
{
	private readonly string connectionString;

	public StockItemsNokExceptionController(IConfiguration configuration)
	{
		this.connectionString = configuration.GetConnectionString("WorldWideImportersDatabase");
	}

	public async Task<ActionResult<IAsyncEnumerable<Model.StockItemListDtoV1>>> GetException()
	{
		IEnumerable<Model.StockItemListDtoV1> response = null;

		using (SqlConnection db = new SqlConnection(this.connectionString))
		{
			response = await db.QueryWithRetryAsync<Model.StockItemListDtoV1>(sql: @"SELECT [StockItemID] as ""ID"", [StockItemName] as ""Name"", [RecommendedRetailPrice], [TaxRate] FROM [Warehouse].[StockItemsException]", commandType: CommandType.Text);
		}

		return this.Ok(response);
	}
}

In “Development” mode the information returned to the caller contains a detailed stack trace that reveals implementation details which are useful for debugging but would also be useful to an attacker.

Developer StockItemsNok Controller Exception page

When not in “Development” mode no additional information is returned (not even a TraceId).

Production StockItemsNok500Controller Exception

The diagnostic stacktrace information logged by the two different controllers was essentially the same

System.Data.SqlClient.SqlException:
   at System.Data.SqlClient.SqlCommand+<>c.<ExecuteDbDataReaderAsync>b__126_0 (System.Data.SqlClient, Version=4.6.1.3, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Threading.Tasks.Task+<>c.<.cctor>b__272_0 (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Threading.ExecutionContext.RunInternal (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Threading.ExecutionContext.RunInternal (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Dapper.SqlMapper+<QueryAsync>d__33`1.MoveNext (Dapper, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null: /_/Dapper/SqlMapper.Async.cs:418)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Polly.Retry.AsyncRetryEngine+<ImplementationAsync>d__0`1.MoveNext (Polly, Version=7.0.0.0, Culture=neutral, PublicKeyToken=c8a3ffc3f8f825cc)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Polly.AsyncPolicy+<ExecuteAsync>d__21`1.MoveNext (Polly, Version=7.0.0.0, Culture=neutral, PublicKeyToken=c8a3ffc3f8f825cc)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at devMobile.WebAPIDapper.Lists.Controllers.StockItemsNokController+<Get500>d__4.MoveNext (ListsClassic, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\BrynLewis\source\repos\WebAPIDapper\Lists\Controllers\14.StockItemsNokController.cs:70)

One customer wanted their client application to display a corporate help desk number for staff to call for support. This information was made configurable

namespace devMobile.WebAPIDapper.Lists
{
	public class ErrorHandlerSettings
	{
		public string Detail { get; set; } = "devMobile Lists Classic API failure";

		public string Title { get; set; } = "System Error";
	}
}
public void ConfigureServices(IServiceCollection services)
{
	services.AddControllers();

	var errorHandlerSettings = Configuration.GetSection(nameof(ErrorHandlerSettings));
	services.Configure<ErrorHandlerSettings>(errorHandlerSettings);

	services.AddResponseCaching();

	services.AddDapperForMSSQL();
#if DAPPER_EXTENSIONS_CACHE_MEMORY
	services.AddDapperCachingInMemory(new MemoryConfiguration
	{
		AllMethodsEnableCache = false
	});
#endif
#if DAPPER_EXTENSIONS_CACHE_REDIS
	services.AddDapperCachingInRedis(new RedisConfiguration
	{
		AllMethodsEnableCache = false,
		KeyPrefix = Configuration.GetConnectionString("RedisKeyPrefix"),
		ConnectionString = Configuration.GetConnectionString("RedisConnection")
	}); 
#endif
	services.AddApplicationInsightsTelemetry();
}
{
 ...
  },
  "ErrorHandlerSettings": {
    "Title": "Webpage has died",
    "Detail": "Something has gone wrong call the help desk on 0800-RebootIt"
  },
...
}
namespace devMobile.WebAPIDapper.Lists.Controllers
{
	using Microsoft.AspNetCore.Mvc;
	using Microsoft.Extensions.Hosting;
	using Microsoft.Extensions.Options;


	[ApiController]
	public class ErrorController : Controller
	{
		private readonly ErrorHandlerSettings errorHandlerSettings;

		public ErrorController(IOptions<ErrorHandlerSettings> errorHandlerSettings)
		{
			this.errorHandlerSettings = errorHandlerSettings.Value;
		}

		[Route("/error")]
		public IActionResult HandleError([FromServices] IHostEnvironment hostEnvironment)
		{
			return Problem(detail: errorHandlerSettings.Detail, title: errorHandlerSettings.Title);
		}
	}
}
StockItemsNok Controller Error page with configurable title and details

Another customer wanted their client application to display a corporate help desk number based on the source hostname.

  • ClientA.SaasApplicationProvider.co.nz
  • ClientB.SaasApplicationProvider.co.nz
  • ClientC.SaasApplicationProvider.co.nz
  • SaasApplication.ClientD.co.nz

This information was also made configurable

namespace devMobile.WebAPIDapper.Lists
{
	using System.Collections.Generic;

	public class UrlSpecificSetting
	{
		public string Title { get; set; } = "";
		
		public string Detail { get; set; } = "";

		public UrlSpecificSetting()
		{
		}

		public UrlSpecificSetting(string title, string detail)
		{
			this.Title = title;
			this.Detail = detail;
		}
	}

	public class ErrorHandlerSettings
	{
		public string Title { get; set; } = "System Error";

		public string Detail { get; set; } = "devMobile Lists Classic API failure";

		public Dictionary<string, UrlSpecificSetting> UrlSpecificSettings { get; set; }

		public ErrorHandlerSettings()
		{
		}

		public ErrorHandlerSettings(string title, string detail, Dictionary<string, UrlSpecificSetting> urlSpecificSettings )
		{
			Title = title;

			Detail = detail;

			UrlSpecificSettings = urlSpecificSettings;
		}
	}
}

We considered storing the title and details message in the database but that approach was discounted as we wanted to minimise dependencies.

{
 ...
  "ErrorHandlerSettings": {
    "Detail": "Default detail",
    "Title": "Default title",
    "UrlSpecificSettings": {
      "localhost": {
        "Title": "Title for localhost",
        "Detail": "Detail for localhost"
      },
      "127.0.0.1": {
        "Title": "Title for 127.0.0.1",
        "Detail": "Detail for 127.0.0.1"
      }
    }
  }
}
namespace devMobile.WebAPIDapper.Lists.Controllers
{
	using Microsoft.AspNetCore.Mvc;
	using Microsoft.Extensions.Hosting;
	using Microsoft.Extensions.Options;


	[ApiController]
	public class ErrorController : Controller
	{
		private readonly ErrorHandlerSettings errorHandlerSettings;

		public ErrorController(IOptions<ErrorHandlerSettings> errorHandlerSettings)
		{
			this.errorHandlerSettings = errorHandlerSettings.Value;
		}

		[Route("/error")]
		public IActionResult HandleError([FromServices] IHostEnvironment hostEnvironment)
		{
			if (!this.errorHandlerSettings.UrlSpecificSettings.ContainsKey(this.Request.Host.Host))
			{
				return Problem(detail: errorHandlerSettings.Detail, title: errorHandlerSettings.Title);
			}

			return Problem(errorHandlerSettings.UrlSpecificSettings[this.Request.Host.Host].Title, errorHandlerSettings.UrlSpecificSettings[this.Request.Host.Host].Detail);
		}
	}
}

The sample configuration has custom title and details text for localhost and 127.0.0.1 with a default title and details text for all other hostnames.

StockItemsNok Controller Error page with 127.0.0.1 specific title and details
StockItemsNok Controller Error page with localhost specific title and details

One customer had a staff member who would take a photo of the client application error page with their mobile and email it to us which made it really easy to track down issues. This was especially usefully as they were in an awkward timezone.

Application Insights TraceId search
Application Insights TraceId search result with exception details

With a customisable error page my approach with the outer try/catch has limited benefit and just adds complexity.

.NET nanoFramework RAK4200 Library Usage

After a two week “soak test” using a Sparkfun Thing Plus ESP32 WROOM and RAK4200 Breakout Board completed with no failures, this final post covers the usage of the RAK4200LoRaWAN-NetNF library in a “real-world” application.

Discovery kit with STM32F769NI MCU and RAK4200 Breakout Board test rig

After a factory reset the DevEUI, JoinEUI (was AppEUI), and AppKey were reset, but the rest of the configuration appeared to be retained.

OK Work Mode: LoRaWAN
Region: AS923
MulticastEnable: false
DutycycleEnable: false
Send_repeat_cnt: 0
Join_mode: OTAA
DevEui: 0000000000000000
AppEui: 0000000000000000
AppKey: 00000000000000000000000000000000
Class: A
Joined Network:false
IsConfirm: unconfirm
AdrEnable: true
EnableRepeaterSupport: false
RX2_CHANNEL_FREQUENCY: 923200000, RX2_CHANNEL_DR:2
RX_WINDOW_DURATION: 3000ms
RECEIVE_DELAY_1: 1000ms
RECEIVE_DELAY_2: 2000ms
JOIN_ACCEPT_DELAY_1: 5000ms
JOIN_ACCEPT_DELAY_2: 6000ms
Current Datarate: 5
Primeval Datarate: 5
ChannelsTxPower: 0
UpLinkCounter: 0
DownLinkCounter: 0

I then ran the RAK4200LoRaWANDeviceClient with DEVICE_DEVEUI_SET (devEui from label on the device), OTAA to configure the AppEui and AppKey and the device connected to The Things Network on the second attempt (typo in the DevEui).

public static void Main()
{
	Result result;

	Debug.WriteLine("devMobile.IoT.RAK4200LoRaWANDeviceClient starting");

	try
	{
		// set GPIO functions for COM2 (this is UART1 on ESP32)
#if ESP32_WROOM
		Configuration.SetPinFunction(Gpio.IO16, DeviceFunction.COM2_TX);
		Configuration.SetPinFunction(Gpio.IO17, DeviceFunction.COM2_RX);
#endif

		Debug.Write("Ports:");
		foreach (string port in SerialPort.GetPortNames())
		{
			Debug.Write($" {port}");
		}
		Debug.WriteLine("");

		using (Rak4200LoRaWanDevice device = new Rak4200LoRaWanDevice())
		{
			result = device.Initialise(SerialPortId, 9600, Parity.None, 8, StopBits.One);
			if (result != Result.Success)
			{
				Debug.WriteLine($"Initialise failed {result}");
				return;
			}

#if CONFIRMED
			device.OnMessageConfirmation += OnMessageConfirmationHandler;
#endif
			device.OnReceiveMessage += OnReceiveMessageHandler;

#if FACTORY_RESET
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} FactoryReset");
			result = device.FactoryReset();
			if (result != Result.Success)
			{
				Debug.WriteLine($"FactoryReset failed {result}");
				return;
			}
#endif

#if DEVICE_DEVEUI_SET
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Device EUI");
			result = device.DeviceEui(Config.devEui);
			if (result != Result.Success)
			{
				Debug.WriteLine($"ADR on failed {result}");
				return;
			}
#endif

#if REGION_SET
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Region{Region}");
			result = device.Region(Region);
			if (result != Result.Success)
			{
				Debug.WriteLine($"Region on failed {result}");
				return;
			}
#endif

#if ADR_SET
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} ADR On");
			result = device.AdrOn();
			if (result != Result.Success)
			{
				Debug.WriteLine($"ADR on failed {result}");
				return;
			}
#endif
#if CONFIRMED
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Confirmed");
			result = device.UplinkMessageConfirmationOn();
			if (result != Result.Success)
			{
				Debug.WriteLine($"Confirm on failed {result}");
				return;
			}
#endif
#if UNCONFIRMED
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Unconfirmed");
			result = device.UplinkMessageConfirmationOff();
			if (result != Result.Success)
			{
				Debug.WriteLine($"Confirm off failed {result}");
				return;
			}
#endif

#if OTAA
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} OTAA");
			result = device.OtaaInitialise(Config.JoinEui, Config.AppKey);
			if (result != Result.Success)
			{
				Debug.WriteLine($"OTAA Initialise failed {result}");
				return;
			}
#endif

#if ABP
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} ABP");
			result = device.AbpInitialise(Config.DevAddress, Config.NwksKey, Config.AppsKey);
			if (result != Result.Success)
			{
				Debug.WriteLine($"ABP Initialise failed {result}");
				return;
			}
#endif

			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Join start Timeout:{JoinTimeOut:hh:mm:ss}");
			result = device.Join(JoinTimeOut);
			if (result != Result.Success)
			{
				Debug.WriteLine($"Join failed {result}");
				return;
			}
			Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Join finish");

			while (true)
			{
#if PAYLOAD_BCD
				Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Send Timeout:{SendTimeout:hh:mm:ss} port:{MessagePort} payload BCD:{PayloadBcd}");
				result = device.Send(MessagePort, PayloadBcd, SendTimeout);
#endif
#if PAYLOAD_BYTES
				Debug.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Send Timeout:{SendTimeout:hh:mm:ss} port:{MessagePort} payload Bytes:{BitConverter.ToString(PayloadBytes)}");
				result = device.Send(MessagePort, PayloadBytes, SendTimeout);
#endif
				if (result != Result.Success)
				{
					Debug.WriteLine($"Send failed {result}");
				}

				Thread.Sleep(new TimeSpan(0, 5, 0));
			}
		}
	}
	catch (Exception ex)
	{
		Debug.WriteLine(ex.Message);
	}
}

After configuring my Discovery kit with STM32F769NI MCU and RAK4200 Breakout Board test rig the RAK4200LoRaWANDeviceClient application could successfully connect to The Things Network with just ST_STM32F769I_DISCOVERY, PAYLOAD_BCD or PAYLOAD_BYTES and CONFIRMED or UNCONFIRMED defined.

Visual Studio Debug output for RAK4200LoRaWANDeviceClient minimal configuration connection
The Things Network “Live Data” for RAK4200LoRaWANDeviceClient minimal configuration connection

One of my client’s products has a configuration mode (button pressed as device starts) which enables a serial port (headers on board + FTDI module) for in field configuration of the onboard RAK4200 module.

.NET nanoFramework RAK4200 Factory Reset

The RAKwireless RAK4200 module has a really useful “reset to factory” AT Command

at+set_config=lora:default_parameters

To see what settings were erased I modified the BreakOutSerial application to send additional AT commands and display the responses.

public static void Main()
{
...
	Debug.WriteLine("devMobile.IoT.LoRaWAN.nanoFramework.RAK4200 BreakoutSerial starting");

	try
	{
		// set GPIO functions for COM2 (this is UART1 on ESP32)
#if ESP32_WROOM
		Configuration.SetPinFunction(Gpio.IO16, DeviceFunction.COM2_TX);
		Configuration.SetPinFunction(Gpio.IO17, DeviceFunction.COM2_RX);
#endif

		Debug.Write("Ports:");
		foreach (string port in SerialPort.GetPortNames())
		{
			Debug.Write($" {port}");
		}
		Debug.WriteLine("");

		using (_SerialPort = new SerialPort(SerialPortId))
		{
			// set parameters
			_SerialPort.BaudRate = 9600;
			//_SerialPort.BaudRate = 115200;
			_SerialPort.Parity = Parity.None;
			_SerialPort.DataBits = 8;
			_SerialPort.StopBits = StopBits.One;
			_SerialPort.Handshake = Handshake.None;
			_SerialPort.NewLine = "\r\n";

			//_SerialPort.ReadBufferSize = 128; 
			//_SerialPort.ReadBufferSize = 256; 
			_SerialPort.ReadBufferSize = 512; 
			//_SerialPort.ReadBufferSize = 1024;
			_SerialPort.ReadTimeout = 1000;

			//_SerialPort.WatchChar = '\n'; // May 2022 WatchChar event didn't fire github issue https://github.com/nanoframework/Home/issues/1035

#if SERIAL_ASYNC_READ
			_SerialPort.DataReceived += SerialDevice_DataReceived;
#endif

			_SerialPort.Open();

			_SerialPort.WatchChar = '\n';
...
			for (int i = 0; i < 5; i++)
			{
				string atCommand;
				atCommand = "at+version";
				//atCommand = "at+set_config=device:uart:1:9600";
				atCommand = "at+get_config=lora:status";
				//atCommand = "at+get_config=device:status";
				//atCommand = "at+get_config=lora:channel";
				//atCommand = "at+help";
				//atCommand = "at+set_config=device:restart";
				//atCommand = "at+set_config=lora:default_parameters";
				Debug.WriteLine("");
				Debug.WriteLine($"{i} TX:{atCommand} bytes:{atCommand.Length}--------------------------------");
				_SerialPort.WriteLine(atCommand);

				Thread.Sleep(5000);
			}
		}
		Debug.WriteLine("Done");
	}
	catch (Exception ex)
	{
		Debug.WriteLine(ex.Message);
	}
}

#if SERIAL_ASYNC_READ
private static void SerialDevice_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
	SerialPort serialPort = (SerialPort)sender;

	switch (e.EventType)
	{
		case SerialData.Chars:
			break;

		case SerialData.WatchChar:
			string response = serialPort.ReadExisting();
			Debug.Write(response);
			break;
		default:
			Debug.Assert(false, $"e.EventType {e.EventType} unknown");
			break;
	}
}
#endif
...
}

After configuring my Sparkfun Thing Plus ESP32 WROOM + RAK42000 Breakout Board test rig the RAK4200LoRaWANDeviceClient application could successfully connect to The Things Network .

The thread '' (0x2) has exited with code 0 (0x0).
devMobile.IoT.LoRaWAN.nanoFramework.RAK4200 BreakoutSerial starting
0 TX:at+get_config=lora:status bytes:25--------------------------------
OK Work Mode: LoRaWAN
Region: AS923
MulticastEnable: false
DutycycleEnable: false
Send_repeat_cnt: 0
Join_mode: OTAA
DevEui:  6..............9
AppEui: 7..............4
AppKey: A.............................9
Class: A
Joined Network:false
IsConfirm: unconfirm
AdrEnable: true
EnableRepeaterSupport: false
RX2_CHANNEL_FREQUENCY: 923200000, RX2_CHANNEL_DR:2
RX_WINDOW_DURATION: 3000ms
RECEIVE_DELAY_1: 1000ms
RECEIVE_DELAY_2: 2000ms
JOIN_ACCEPT_DELAY_1: 5000ms
JOIN_ACCEPT_DELAY_2: 6000ms
Current Datarate: 5
Primeval Datarate: 5
ChannelsTxPower: 0
UpLinkCounter: 0
DownLinkCounter: 0

I then reset the RAK4200 device to “factory defaults”

The thread '<No Name>' (0x2) has exited with code 0 (0x0).
devMobile.IoT.LoRaWAN.nanoFramework.RAK4200 BreakoutSerial starting
Ports: COM5 COM6

0 TX:at+set_config=lora:default_parameters bytes:37--------------------------------
OK

The testrig would no longer connect as the device and network settings were invalid.

The thread '<No Name>' (0x2) has exited with code 0 (0x0).
devMobile.IoT.LoRaWAN.nanoFramework.RAK4200 BreakoutSerial starting
Ports: COM5 COM6

0 TX:at+get_config=lora:status bytes:25--------------------------------
OK Work Mode: LoRaWAN
Region: AS923
MulticastEnable: false
DutycycleEnable: false
Send_repeat_cnt: 0
Join_mode: OTAA
DevEui: 0000000000000000
AppEui: 0000000000000000
AppKey: 00000000000000000000000000000000
Class: A
Joined Network:false
IsConfirm: unconfirm
AdrEnable: true
EnableRepeaterSupport: false
RX2_CHANNEL_FREQUENCY: 923200000, RX2_CHANNEL_DR:2
RX_WINDOW_DURATION: 3000ms
RECEIVE_DELAY_1: 1000ms
RECEIVE_DELAY_2: 2000ms
JOIN_ACCEPT_DELAY_1: 5000ms
JOIN_ACCEPT_DELAY_2: 6000ms
Current Datarate: 5
Primeval Datarate: 5
ChannelsTxPower: 0
UpLinkCounter: 0
DownLinkCounter: 0

To “restore” the device configuration I ran the RAK4200LoRaWANDeviceClient application with DEVICE_DEVEUI_SET, OTAA, UNCONFIRMED, REGION_SET and ADR_SET defined.

The thread '<No Name>' (0x2) has exited with code 0 (0x0).
devMobile.IoT.LoRaWAN.nanoFramework.RAK4200 BreakoutSerial starting
Ports: COM2 COM3

0 TX:at+get_config=lora:status bytes:25--------------------------------
OK Work Mode: LoRaWAN
Region: AS923
MulticastEnable: false
DutycycleEnable: false
Send_repeat_cnt: 0
Join_mode: OTAA
DevEui: 6..............9
AppEui: 7.............4
AppKey: A.............................9
Class: A
Joined Network:false
IsConfirm: unconfirm
AdrEnable: true
EnableRepeaterSupport: false
RX2_CHANNEL_FREQUENCY: 923200000, RX2_CHANNEL_DR:2
RX_WINDOW_DURATION: 3000ms
RECEIVE_DELAY_1: 1000ms
RECEIVE_DELAY_2: 2000ms
JOIN_ACCEPT_DELAY_1: 5000ms
JOIN_ACCEPT_DELAY_2: 6000ms
Current Datarate: 5
Primeval Datarate: 5
ChannelsTxPower: 0
UpLinkCounter: 0
DownLinkCounter: 0

The testrig would then successfully connect to The Things Network. When the testrig was power cycled the device the configuration was retained.

.NET nanoFramework RAK4200 LoRaWAN library ABP Join

After getting my RAKwireless RAK4200 module to reliably connect to The Things Network(TTN) using Over The Air Activation(OTAA) the next step was to built an Activation By Personalisation(ABP) sample application.

I modified the NetworkJoinOTAA sample(based on the asynchronous version of BreakOutSerial) to send the required sequence of AT commands and display the responses.

namespace devMobile.IoT.LoRaWAN.nanoFramework.RAK4200
{
   using System;
   using System.Diagnostics;
   using System.IO.Ports;
   using System.Text;
   using System.Threading;
#if ESP32_WROOM
   using global::nanoFramework.Hardware.Esp32; ///need NuGet nanoFramework.Hardware.Esp32
#endif

   public class Program
   {
#if ST_STM32F769I_DISCOVERY
      private const string SerialPortId = "COM6";
#endif
#if ESP32_WROOM
      private const string SerialPortId = "COM2";
#endif
      private const string DevEui = "...";
      private const string AppEui = "...";
      private const string AppKey = "...";
      private const byte MessagePort = 1;
      private const string Payload = "01020304"; // Is AQIDBA==

      public static void Main()
      {
         string response;

         Debug.WriteLine("devMobile.IoT.Rak4200.NetworkJoinOTAA starting");

         try
         {
#if ESP32_WROOM
            Configuration.SetPinFunction(Gpio.IO16, DeviceFunction.COM2_TX);
            Configuration.SetPinFunction(Gpio.IO17, DeviceFunction.COM2_RX);
#endif

            Debug.Write("Ports:");
            foreach (string port in SerialPort.GetPortNames())
            {
               Debug.Write($" {port}");
            }
            Debug.WriteLine("");

            using (SerialPort serialDevice = new SerialPort(SerialPortId))
            {
               // set parameters
               serialDevice.BaudRate = 9600;
               //_SerialPort.BaudRate = 115200;
               serialDevice.Parity = Parity.None;
               serialDevice.StopBits = StopBits.One;
               serialDevice.Handshake = Handshake.None;
               serialDevice.DataBits = 8;

               serialDevice.ReadTimeout = 10000;
               //serialDevice.ReadBufferSize = 128; 
               //serialDevice.ReadBufferSize = 256; 
               serialDevice.ReadBufferSize = 512;
               //serialDevice.ReadBufferSize = 1024;

               serialDevice.NewLine = "\r\n";

               serialDevice.DataReceived += SerialDevice_DataReceived;

               serialDevice.Open();

               serialDevice.WatchChar = '\n';

               // clear out the RX buffer
               serialDevice.ReadExisting();
               response = serialDevice.ReadExisting();
               Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");
               Thread.Sleep(500);

               // Set the Working mode to LoRaWAN
               Debug.WriteLine("lora:work_mode:0");
               serialDevice.WriteLine("at+set_config=lora:work_mode:0");
               Thread.Sleep(1500);

               // Set the JoinMode
               Debug.WriteLine("lora:join_mode");
               serialDevice.WriteLine("at+set_config=lora:join_mode:0");
               Thread.Sleep(500);

               // Set the Class
               Debug.WriteLine("lora:class");
               serialDevice.WriteLine("at+set_config=lora:class:0");
               Thread.Sleep(500);

               // Set the Region to AS923
               Debug.WriteLine("lora:region");
               serialDevice.WriteLine("at+set_config=lora:region:AS923");
               Thread.Sleep(500);

               // Set the devEUI
               Debug.WriteLine("lora:dev_eui");
               serialDevice.WriteLine($"at+set_config=lora:dev_eui:{DevEui}");
               Thread.Sleep(500);

               // Set the appEUI
               Debug.WriteLine("lora:app_eui");
               serialDevice.WriteLine($"at+set_config=lora:app_eui:{AppEui}");
               Thread.Sleep(500);

               // Set the appKey
               Debug.WriteLine("lora:app_key");
               serialDevice.WriteLine($"at+set_config=lora:app_key:{AppKey}");
               Thread.Sleep(500);

               // Set the Confirm flag
               Debug.WriteLine("lora:confirm");
               serialDevice.WriteLine("at+set_config=lora:confirm:0");
               Thread.Sleep(500);

               Debug.WriteLine("lora:adr");
               serialDevice.WriteLine("at+set_config=lora:adr:1");
               Thread.Sleep(500);

               // Join the network
               Debug.WriteLine("at+join");
               serialDevice.WriteLine("at+join");
               Thread.Sleep(10000);

               byte counter = 1;
               while (true)
               {
                  // Send the BCD messages
                  string payload = $"{Payload}{counter:X2}";
                  Debug.WriteLine($"at+send=lora:{MessagePort}:{payload}");
                  serialDevice.WriteLine($"at+send=lora:{MessagePort}:{payload}");

                  counter += 1;

                  Thread.Sleep(300000);
               }
            }
         }
         catch (Exception ex)
         {
            Debug.WriteLine(ex.Message);
         }
      }

      private static void SerialDevice_DataReceived(object sender, SerialDataReceivedEventArgs e)
      {
         SerialPort serialPort = (SerialPort)sender;
         string response;

         switch (e.EventType)
         {
            case SerialData.Chars:
               break;

            case SerialData.WatchChar:
               response = serialPort.ReadExisting();
               Debug.Write(response);
               break;
            default:
               Debug.Assert(false, $"e.EventType {e.EventType} unknown");
               break;
         }
      }
   }
}

The NetworkJoinABP application assumes that all of the AT commands succeed.

TTN Console live data tab connection process
Visual Studio Output windows displaying connection process and a D2C message

I have been using at+set_config=lora:default_parameters set the WisDuo RAK4200 back to factory settings so I can figure out what settings are persisted by a “at+set_config=device:restart” and which ones need to be set every time the application is run.

Smartish Edge Camera – Azure IoT Updateable Properties (not persisted)

This post builds on my Smartish Edge Camera -Azure IoT Direct Methods post adding two updateable properties for the image capture and processing timer the due and period values. The two properties can be updated together or independently but the values are not persisted.

When I was searching for answers I found this code in many posts and articles but it didn’t really cover my scenario.

private static async Task OnDesiredPropertyChanged(TwinCollection desiredProperties, 
  object userContext)
{
   Console.WriteLine("desired property chPleange:");
   Console.WriteLine(JsonConvert.SerializeObject(desiredProperties));
   Console.WriteLine("Sending current time as reported property");
   TwinCollection reportedProperties = new TwinCollection
   {
       ["DateTimeLastDesiredPropertyChangeReceived"] = DateTime.Now
   };

    await Client.UpdateReportedPropertiesAsync(reportedProperties).ConfigureAwait(false);
}

When AZURE_DEVICE_PROPERTIES is defined in the SmartEdgeCameraAzureIoTService project properties the device reports a number of properties on startup and SetDesiredPropertyUpdateCallbackAsync is used to configure the method called whenever the client receives a state update(desired or reported) from the Azure IoT Hub.

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
	_logger.LogInformation("Azure IoT Smart Edge Camera Service starting");

	try
	{
#if AZURE_IOT_HUB_CONNECTION
		_deviceClient = await AzureIoTHubConnection();
#endif
#if AZURE_IOT_HUB_DPS_CONNECTION
		_deviceClient = await AzureIoTHubDpsConnection();
#endif

#if AZURE_DEVICE_PROPERTIES
		_logger.LogTrace("ReportedPropeties upload start");

		TwinCollection reportedProperties = new TwinCollection();

		reportedProperties["OSVersion"] = Environment.OSVersion.VersionString;
		reportedProperties["MachineName"] = Environment.MachineName;
		reportedProperties["ApplicationVersion"] = Assembly.GetAssembly(typeof(Program)).GetName().Version;
		reportedProperties["ImageTimerDue"] = _applicationSettings.ImageTimerDue;
		reportedProperties["ImageTimerPeriod"] = _applicationSettings.ImageTimerPeriod;
		reportedProperties["YoloV5ModelPath"] = _applicationSettings.YoloV5ModelPath;

		reportedProperties["PredictionScoreThreshold"] = _applicationSettings.PredictionScoreThreshold;
		reportedProperties["PredictionLabelsOfInterest"] = _applicationSettings.PredictionLabelsOfInterest;
		reportedProperties["PredictionLabelsMinimum"] = _applicationSettings.PredictionLabelsMinimum;

		await _deviceClient.UpdateReportedPropertiesAsync(reportedProperties, stoppingToken);

		_logger.LogTrace("ReportedPropeties upload done");
#endif

		_logger.LogTrace("YoloV5 model setup start");
		_scorer = new YoloScorer<YoloCocoP5Model>(_applicationSettings.YoloV5ModelPath);
		_logger.LogTrace("YoloV5 model setup done");

		_ImageUpdatetimer = new Timer(ImageUpdateTimerCallback, null, _applicationSettings.ImageTimerDue, _applicationSettings.ImageTimerPeriod);

		await _deviceClient.SetMethodHandlerAsync("ImageTimerStart", ImageTimerStartHandler, null);
		await _deviceClient.SetMethodHandlerAsync("ImageTimerStop", ImageTimerStopHandler, null);
		await _deviceClient.SetMethodDefaultHandlerAsync(DefaultHandler, null);

		await _deviceClient.SetDesiredPropertyUpdateCallbackAsync(OnDesiredPropertyChangedAsync, null);

		try
		{
			await Task.Delay(Timeout.Infinite, stoppingToken);
		}
		catch (TaskCanceledException)
		{
			_logger.LogInformation("Application shutown requested");
		}
	}
	catch (Exception ex)
	{
		_logger.LogError(ex, "Application startup failure");
	}
	finally
	{
		_deviceClient?.Dispose();
	}

	_logger.LogInformation("Azure IoT Smart Edge Camera Service shutdown");
}

// Lots of other code here

private async Task OnDesiredPropertyChangedAsync(TwinCollection desiredProperties, object userContext)
{
	TwinCollection reportedProperties = new TwinCollection();

	_logger.LogInformation("OnDesiredPropertyChanged handler");

	// NB- This approach does not save the ImageTimerDue or ImageTimerPeriod, a stop/start with return to appsettings.json configuration values. If only
	// one parameter specified other is default from appsettings.json. If timer settings changed I think they won't take
	// effect until next time Timer fires.

	try
	{
		// Check to see if either of ImageTimerDue or ImageTimerPeriod has changed
		if (!desiredProperties.Contains("ImageTimerDue") && !desiredProperties.Contains("ImageTimerPeriod"))
		{
			_logger.LogInformation("OnDesiredPropertyChanged neither ImageTimerDue or ImageTimerPeriod present");
			return;
		}

		TimeSpan imageTimerDue = _applicationSettings.ImageTimerDue;

		// Check that format of ImageTimerDue valid if present
		if (desiredProperties.Contains("ImageTimerDue"))
		{
			if (TimeSpan.TryParse(desiredProperties["ImageTimerDue"].Value, out imageTimerDue))
			{
				reportedProperties["ImageTimerDue"] = imageTimerDue;
			}
			else
			{
				_logger.LogInformation("OnDesiredPropertyChanged ImageTimerDue invalid");
				return;
			}
		}

		TimeSpan imageTimerPeriod = _applicationSettings.ImageTimerPeriod;

		// Check that format of ImageTimerPeriod valid if present
		if (desiredProperties.Contains("ImageTimerPeriod"))
		{
			if (TimeSpan.TryParse(desiredProperties["ImageTimerPeriod"].Value, out imageTimerPeriod))
			{
				reportedProperties["ImageTimerPeriod"] = imageTimerPeriod;
			}
			else
			{
				_logger.LogInformation("OnDesiredPropertyChanged ImageTimerPeriod invalid");
				return;
			}
		}

		_logger.LogInformation("Desired Due:{0} Period:{1}", imageTimerDue, imageTimerPeriod);

		if (!_ImageUpdatetimer.Change(imageTimerDue, imageTimerPeriod))
		{
			_logger.LogInformation("Desired Due:{0} Period:{1} failed", imageTimerDue, imageTimerPeriod);
		}

		await _deviceClient.UpdateReportedPropertiesAsync(reportedProperties);
	}
	catch (Exception ex)
	{
		_logger.LogError(ex, "OnDesiredPropertyChangedAsync handler failed");
	}
}

The TwinCollection desiredProperties is checked for ImageTimerDue and ImageTimerPeriod properties and if either of these are present and valid the Timer.Change method is called.

The AzureMLMetSmartEdgeCamera supports both Azure IoT Hub and Azure IoT Central so I have included images from Azure IoT Explorer and my Azure IoT Central Templates.

SmartEdge Camera Device Twin properties in Azure IoT Explorer

When I modified, then saved the Azure IoT Hub Device Twin desired properties JavaScript Object Notation(JSON) in Azure IoT Hub Explorer the method configured with SetDesiredPropertyUpdateCallbackAsync was invoked on the device.

In Azure IoT Central I added two Capabilities to the device template, the time properties ImageTimerDue, and ImageTimerPeriod.

Azure IoT Central SmartEdgeCamera Device template capabilities

I added a View to the template so the two properties could be changed (I didn’t configure either as required)

Azure IoT Central SmartEdgeCamera Device Default view designer

In the “Device Properties”, “Operation Tab” when I changed the ImageTimerDue and/or ImageTimerPeriod there was visual feedback that there was an update in progress.

Azure IoT Central SmartEdgeCamera Device Properties update start

Then on the device the SmartEdgeCameraAzureIoTService the method configured with SetDesiredPropertyUpdateCallbackAsync was invoked on the device.

SmartEdge Camera Console application displaying updated properties

Once the properties have been updated on the device the UpdateReportedPropertiesAsync method is called

Then a message with the updated property values from the device was visible in the telemetry

Azure IoT Central SmartEdgeCamera Device Properties update done

Then finally the “Operation Tab” displayed a visual confirmation that the value(s) had been updated.

.NET nanoFramework RAK4200 LoRaWAN library OTAA Join

When I first tried Over The Air Activation(OTAA) to connect to The Things Network(TTN) with my RAKwireless RAK4200 module it didn’t work. I built another test harness with an FTDI module so I could send AT commands with the RAK Serial Port Tool to my RAK4200 module.

RAK4200 -> FTDI -? PC test harness

The default baud rate is 115200 but I sent “at+set_config=device:uart:1:9600” to the RAK4200 module.

RAK Serial Port Tool initiating at+join command

With the RAK Serial Port Tool I could get the RAK4200 connected to TTN and send unconfirmed messages. The sequence of commands I used was

at+set_config=lora:join_mode:0
at+set_config=lora:class:0
at+set_config=lora:region:AS923
at+set_config=lora:dev_eui:XXXX
at+set_config=lora:app_eui:XXXX
at+set_config=lora:app_key:XXXX
at+set_config=device:restart
at+join
at+send=lora:2:48656c6c6f204c6f526157414e

I then returned to my STM32F769I Discovery, RAK4200 Breakoutboard, Seeedstudio Grove Base Shield for Arduino and a Seeedstudio Grove-4 pin Female Jumper to Grove 4 pin Conversion Cable based test harness.

RAK4200, STM32F769I Discovery test harness

I modified the NetworkJoinOTAA sample(based on the asynchronous version of BreakOutSerial) to send the same sequence of AT commands and display the responses.

namespace devMobile.IoT.LoRaWAN.nanoFramework.RAK4200
{
   using System;
	using System.Diagnostics;
   using System.IO.Ports;
   using System.Threading;

   public class Program
	{
      private const string SerialPortId = "COM6";
      private const string DevEui = "...";
      private const string AppEui = "...";
      private const string AppKey = "...";
      private const byte MessagePort = 1;
      private const string Payload = "48656c6c6f204c6f526157414e"; // Hello LoRaWAN

      public static void Main()
      {
         string response;

         Debug.WriteLine("devMobile.IoT.Rak4200.NetworkJoinOTAA starting");

         Debug.Write("Ports:");
         foreach (string port in SerialPort.GetPortNames())
         {
            Debug.Write($" {port}");
         }
         Debug.WriteLine("");

         try
         {
            using (SerialPort serialDevice = new SerialPort(SerialPortId))
            {
               // set parameters
               serialDevice.BaudRate = 9600;
               //_SerialPort.BaudRate = 115200;
               serialDevice.Parity = Parity.None;
               serialDevice.StopBits = StopBits.One;
               serialDevice.Handshake = Handshake.None;
               serialDevice.DataBits = 8;

               serialDevice.ReadTimeout = 10000;

               serialDevice.NewLine = "\r\n";

               serialDevice.DataReceived += SerialDevice_DataReceived;

               serialDevice.Open();

               serialDevice.WatchChar = '\n';

               // clear out the RX buffer
               serialDevice.ReadExisting();
               response = serialDevice.ReadExisting();
               Debug.WriteLine($"Response :{response.Trim()} bytes:{response.Length}");
               Thread.Sleep(500);

               // Set the Working mode to LoRaWAN
               Console.WriteLine("lora:work_mode:0");
               serialDevice.WriteLine("at+set_config=lora:work_mode:0");

               // Set the JoinMode
               Console.WriteLine("lora:join_mode");
               serialDevice.WriteLine("at+set_config=lora:join_mode:0");
               Thread.Sleep(500);

               // Set the Class
               Console.WriteLine("lora:class");
               serialDevice.WriteLine("at+set_config=lora:class:0");
               Thread.Sleep(500);

               // Set the Region to AS923
               Console.WriteLine("lora:region:AS923");
               serialDevice.WriteLine("at+set_config=lora:region:AS923");
               Thread.Sleep(500);

               // Set the devEUI
               Console.WriteLine("lora:dev_eui:{DevEui}");
               serialDevice.WriteLine($"at+set_config=lora:dev_eui:{DevEui}");
               Thread.Sleep(500);

               // Set the appEUI
               Console.WriteLine("lora:app_eui:{AppEui}");
               serialDevice.WriteLine($"at+set_config=lora:app_eui:{AppEui}");
               Thread.Sleep(500);

               // Set the appKey
               Console.WriteLine("lora:app_key:{AppKey}");
               serialDevice.WriteLine($"at+set_config=lora:app_key:{AppKey}");
               Thread.Sleep(500);

               // Set the Confirm flag
               Console.WriteLine("lora:confirm:0");
               serialDevice.WriteLine("at+set_config=lora:confirm:0");
               Thread.Sleep(500);

               // Reset the device
               Console.WriteLine("device:restart");
               serialDevice.WriteLine($"at+set_config=device:restart");
               Thread.Sleep(10000);

               // Join the network
               Console.WriteLine("at+join");
               serialDevice.WriteLine("at+join");
               Thread.Sleep(10000);

               while (true)
               {
                  // Send the BCD messages
                  Console.WriteLine("lora:{MessagePort}:{Payload}");
                  serialDevice.WriteLine($"at+send=lora:{MessagePort}:{Payload}");

                  Thread.Sleep(20000);
               }
            }
         }
         catch (Exception ex)
         {
            Debug.WriteLine(ex.Message);
         }
      }

      private static void SerialDevice_DataReceived(object sender, SerialDataReceivedEventArgs e)
      {
         SerialPort serialPort = (SerialPort)sender;
         string response;

         switch (e.EventType)
         {
            case SerialData.Chars:
               break;

            case SerialData.WatchChar:
               response = serialPort.ReadExisting();
               Debug.Write(response);
               break;
            default:
               Debug.Assert(false, $"e.EventType {e.EventType} unknown");
               break;
         }
      }
   }
}

The NetworkJoinOTAA application assumes that all of the AT commands succeed

Visual Studio Output windows displaying connection process and a D2C message
TTN Console live data tab connection process
TTN Console live messaging tab C2D message

I need to find a way to set the RAK4200 back to factory settings so I can figure out what settings are persisted by the “at+set_config=device:restart” and which ones need to be set every time the application is run.