Random wanderings through Microsoft Azure esp. PaaS plumbing, the IoT bits, AI on Micro controllers, AI on Edge Devices, .NET nanoFramework, .NET Core on *nix and ML.NET+ONNX
I configured the demonstrationUltralytics YoloV8object detection(yolov8s.onnx) console application to process a 1920×1080 image from a security camera on my desktop development box (13th Gen Intel(R) Core(TM) i7-13700 2.10 GHz with 32.0 GB)
Object Detection sample application running on my development box
A Seeedstudio reComputerJ3011 uses a Nividia Jetson Orin 8G and looked like a cost-effective platform to explore how a dedicated Artificial Intelligence (AI) co-processor could reduce inferencing times.
To establish a “baseline” I “published” the demonstration application on my development box which created a folder with all the files required to run the application on the Seeedstudio reComputerJ3011 ARM64 CPU. I had to manually merge the “User Secrets” and appsettings.json files so the camera connection configuration was correct.
The runtimes folder contained a number of folders with the native runtime files for the supported Open Neural Network Exchange(ONNX) platforms
This Nividia Jetson Orin ARM64 CPU requires the linux-arm64 ONNX runtime which was “automagically” detected. (in previous versions of ML.Net the native runtime had to be copied to the execution directory)
Object Detection sample application running on my Seeedstudio reComputer J3011
When I averaged the pre-processing, inferencing and post-processing times for both devices over 20 executions my development box was much faster which was not a surprise. Though the reComputer J3011 post processing times were a bit faster than I was expecting
ARM64 CPU Preprocess 0.05s Inference 0.31s Postprocess 0.05
Important: make sure you setup the I2C pins especially on ESP32 Devices before creating the I2cDevice,
SHT20 +STM32F769 Discovery test rig
The .NET nanoFramework device libraries use a TryGet… pattern to retrieve sensor values, this library throws an exception if reading a sensor value fails. I’m not certain which approach is “better” as reading the Seeedstudio Grove – Laser PM2.5 Dust Sensor has never failed. The only time reading the “values” buffer failed was when I unplugged the device which I think is “exceptional”.
//---------------------------------------------------------------------------------
// Copyright (c) April 2023, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// nanoff --target ST_STM32F769I_DISCOVERY --update
// nanoff --platform ESP32 --serialport COM7 --update
//
//---------------------------------------------------------------------------------
#define ST_STM32F769I_DISCOVERY
//#define SPARKFUN_ESP32_THING_PLUS
namespace devMobile.IoT.Device.SeeedstudioHM3301
{
using System;
using System.Device.I2c;
using System.Threading;
#if SPARKFUN_ESP32_THING_PLUS
using nanoFramework.Hardware.Esp32;
#endif
class Program
{
static void Main(string[] args)
{
const int busId = 1;
Thread.Sleep(5000);
#if SPARKFUN_ESP32_THING_PLUS
Configuration.SetPinFunction(Gpio.IO23, DeviceFunction.I2C1_DATA);
Configuration.SetPinFunction(Gpio.IO22, DeviceFunction.I2C1_CLOCK);
#endif
I2cConnectionSettings i2cConnectionSettings = new(busId, SeeedstudioHM3301.DefaultI2cAddress);
using I2cDevice i2cDevice = I2cDevice.Create(i2cConnectionSettings);
{
using (SeeedstudioHM3301 seeedstudioHM3301 = new SeeedstudioHM3301(i2cDevice))
{
while (true)
{
SeeedstudioHM3301.ParticulateMeasurements particulateMeasurements = seeedstudioHM3301.Read();
Console.WriteLine($"Standard PM1.0: {particulateMeasurements.Standard.PM1_0} ug/m3 PM2.5: {particulateMeasurements.Standard.PM2_5} ug/m3 PM10.0: {particulateMeasurements.Standard.PM10_0} ug/m3 ");
Console.WriteLine($"Atmospheric PM1.0: {particulateMeasurements.Atmospheric.PM1_0} ug/m3 PM2.5: {particulateMeasurements.Atmospheric.PM2_5} ug/m3 PM10.0: {particulateMeasurements.Standard.PM10_0} ug/m3");
// Always 0, checked payload so not a conversion issue. will check in Seeedstudio forums
// Console.WriteLine($"Count 0.3um: {particulateMeasurements.Count.Diameter0_3}/l 0.5um: {particulateMeasurements.Count.Diameter0_5} /l 1.0um : {particulateMeasurements.Count.Diameter1_0}/l 2.5um : {particulateMeasurements.Count.Diameter2_5}/l 5.0um : {particulateMeasurements.Count.Diameter5_0}/l 10.0um : {particulateMeasurements.Count.Diameter10_0}/l");
Thread.Sleep(new TimeSpan(0,1,0));
}
}
}
}
}
}
I’m going to soak test the library for a week to check that is working okay, then most probably refactor the code so it can be added to the nanoFramework IoT.Device Library repository.
The unpacking of the value standard particulate, particulate count and particle count values is fairly repetitive, but I will fix it in the next version.
Visual Studio 2022 Debug output
The checksum calculation isn’t great even a simple cyclic redundancy check(CRC) would be an improvement on summing the 28 bytes of the payload.
Even though SPI is an industry standard there are often subtle differences which need to be taken into account when reading from/writing to registers. The DW1000 has a static “Device Identifier” which I used to debug my “proof of concept” code.
DW1000 Datasheet Register Map documentation for Register 0x00
The DeviceSPI program reads register 0x00 and then displays the decoded payload.
public class Program
{
#if MAKERFABS_ESP32UWB
private const int SpiBusId = 1;
private const int chipSelectLine = Gpio.IO04;
#endif
public static void Main()
{
Thread.Sleep(5000);
Debug.WriteLine("devMobile.IoT.Dw1000.ShieldSPI starting");
try
{
#if MAKERFABS_ESP32UWB
Configuration.SetPinFunction(Gpio.IO19, DeviceFunction.SPI1_MISO);
Configuration.SetPinFunction(Gpio.IO23, DeviceFunction.SPI1_MOSI);
Configuration.SetPinFunction(Gpio.IO18, DeviceFunction.SPI1_CLOCK);
#endif
var settings = new SpiConnectionSettings(SpiBusId, chipSelectLine)
{
ClockFrequency = 2000000,
Mode = SpiMode.Mode0,
};
using (SpiDevice device = SpiDevice.Create(settings))
{
Thread.Sleep(500);
while (true)
{
/*
byte[] writeBuffer = new byte[] { 0x0, 0x0, 0x0, 0x0, 0x0 }; // 0x0 = DEV_ID
byte[] readBuffer = new byte[writeBuffer.Length];
device.TransferFullDuplex(writeBuffer, readBuffer); // 15, 48, 1, 202, 222
*/
byte[] writeBuffer = new byte[] { 0x0 }; // 0x0 = DEV_ID
byte[] readBuffer = new byte[5];
device.TransferFullDuplex(writeBuffer, readBuffer); // 15, 48, 1, 202, 222
uint ridTag = (uint)(readBuffer[4]<< 8 | readBuffer[3]);
byte model = readBuffer[2];
byte ver = (byte)(readBuffer[1] >> 4);
byte rev = (byte)(readBuffer[1] & 0x0f);
Debug.WriteLine(String.Format($"RIDTAG 0x{ridTag:X2} MODEL 0x{model:X2} VER 0X{ver:X2} REV 0x{rev:X2}"));
Thread.Sleep(10000);
}
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
}
Visual Studio 2022 Debug window displaying the decoded value from Register 0x0
The DW1000 User Manual is > 240 pages, with roughly 140 pages of detailed documentation about the DW1000 register set so progress will be slow.
I have included sample application to show how to use the library
namespace devMobile.IoT.NetCore.GroveBaseHat
{
using System;
using System.Device.I2c;
using System.Threading;
class Program
{
static void Main(string[] args)
{
// bus id on the raspberry pi 3
const int busId = 1;
I2cConnectionSettings i2cConnectionSettings = new(busId, AnalogPorts.DefaultI2cAddress);
using (I2cDevice i2cDevice = I2cDevice.Create(i2cConnectionSettings))
using (AnalogPorts AnalogPorts = new AnalogPorts(i2cDevice))
{
Console.WriteLine($"{DateTime.Now:HH:mm:SS} Version:{AnalogPorts.Version()}");
Console.WriteLine();
double powerSupplyVoltage = AnalogPorts.PowerSupplyVoltage();
Console.WriteLine($"{DateTime.Now:HH:mm:SS} Power Supply Voltage:{powerSupplyVoltage:F2}v");
while (true)
{
double value = AnalogPorts.Read(AnalogPorts.AnalogPort.A0);
double rawValue = AnalogPorts.ReadRaw(AnalogPorts.AnalogPort.A0);
double voltageValue = AnalogPorts.ReadVoltage(AnalogPorts.AnalogPort.A0);
Console.WriteLine($"{DateTime.Now:HH:mm:SS} Value:{value:F2} Raw:{rawValue:F2} Voltage:{voltageValue:F2}v");
Console.WriteLine();
Thread.Sleep(1000);
}
}
}
}
}
The GROVE_BASE_HAT_RPI and GROVE_BASE_HAT_RPI_ZERO are used to specify the number of available analog ports.
//---------------------------------------------------------------------------------
// Copyright (c) September 2021, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// Must have one of following options defined in the project\build definitions
// PAYLOAD_BCD or PAYLOAD_BYTES
// OTAA or ABP
//
// Optional definitions
// CONFIRMED For confirmed messages
// DEVEUI_SET
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.LoRaWAN.NetCore.RAK3172
{
using System;
using System.IO.Ports;
using System.Threading;
public class Program
{
private const string SerialPortId = "/dev/ttyS0";
private const LoRaClass Class = LoRaClass.A;
private const string Band = "8-1";
private const byte MessagePort = 10;
private static readonly TimeSpan MessageSendTimerDue = new TimeSpan(0, 0, 15);
private static readonly TimeSpan MessageSendTimerPeriod = new TimeSpan(0, 5, 0);
private static Timer MessageSendTimer ;
private const int JoinRetryAttempts = 2;
private const int JoinRetryIntervalSeconds = 10;
#if PAYLOAD_BCD
private const string PayloadBcd = "48656c6c6f204c6f526157414e"; // Hello LoRaWAN in BCD
#endif
#if PAYLOAD_BYTES
private static readonly byte[] PayloadBytes = { 0x48, 0x65 , 0x6c, 0x6c, 0x6f, 0x20, 0x4c, 0x6f, 0x52, 0x61, 0x57, 0x41, 0x4e}; // Hello LoRaWAN in bytes
#endif
public static void Main()
{
Result result;
Console.WriteLine("devMobile.IoT.LoRaWAN.NetCore.RAK3172 RAK3712LoRaWANDeviceClient starting");
Console.WriteLine($"Serial ports:{String.Join(",", SerialPort.GetPortNames())}");
try
{
using (Rak3172LoRaWanDevice device = new Rak3172LoRaWanDevice())
{
result = device.Initialise(SerialPortId, 9600, Parity.None, 8, StopBits.One);
if (result != Result.Success)
{
Console.WriteLine($"Initialise failed {result}");
return;
}
MessageSendTimer = new Timer(SendMessageTimerCallback, device,Timeout.Infinite, Timeout.Infinite);
device.OnJoinCompletion += OnJoinCompletionHandler;
device.OnReceiveMessage += OnReceiveMessageHandler;
#if CONFIRMED
device.OnMessageConfirmation += OnMessageConfirmationHandler;
#endif
#if DEVEUI_SET
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} DevEUI {Config.devEui}");
result = device.DeviceEui(Config.devEui);
if (result != Result.Success)
{
Console.WriteLine($"DevEUI failed {result}");
return;
}
#endif
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Class {Class}");
result = device.Class(Class);
if (result != Result.Success)
{
Console.WriteLine($"Class failed {result}");
return;
}
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Band {Band}");
result = device.Band(Band);
if (result != Result.Success)
{
Console.WriteLine($"Region failed {result}");
return;
}
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} ADR On");
result = device.AdrOn();
if (result != Result.Success)
{
Console.WriteLine($"ADR on failed {result}");
return;
}
#if CONFIRMED
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Confirmed");
result = device.UplinkMessageConfirmationOn();
if (result != Result.Success)
{
Console.WriteLine($"Confirm on failed {result}");
return;
}
#else
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Unconfirmed");
result = device.UplinkMessageConfirmationOff();
if (result != Result.Success)
{
Console.WriteLine($"Confirm off failed {result}");
return;
}
#endif
#if OTAA
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} OTAA");
result = device.OtaaInitialise(Config.JoinEui, Config.AppKey);
if (result != Result.Success)
{
Console.WriteLine($"OTAA Initialise failed {result}");
return;
}
#endif
#if ABP
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} ABP");
result = device.AbpInitialise(Config.DevAddress, Config.NwksKey, Config.AppsKey);
if (result != Result.Success)
{
Console.WriteLine($"ABP Initialise failed {result}");
return;
}
#endif
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Join start");
result = device.Join(JoinRetryAttempts, JoinRetryIntervalSeconds);
if (result != Result.Success)
{
Console.WriteLine($"Join failed {result}");
return;
}
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Join started");
Thread.Sleep(Timeout.Infinite);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
private static void OnJoinCompletionHandler(bool result)
{
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Join finished:{result}");
if (result)
{
MessageSendTimer.Change(MessageSendTimerDue, MessageSendTimerPeriod);
}
}
private static void SendMessageTimerCallback(object state)
{
Rak3172LoRaWanDevice device = (Rak3172LoRaWanDevice)state;
#if PAYLOAD_BCD
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} port:{MessagePort} payload BCD:{PayloadBcd}");
Result result = device.Send(MessagePort, PayloadBcd );
#endif
#if PAYLOAD_BYTES
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} port:{MessagePort} payload bytes:{Rak3172LoRaWanDevice.BytesToBcd(PayloadBytes)}");
Result result = device.Send(MessagePort, PayloadBytes);
#endif
if (result != Result.Success)
{
Console.WriteLine($"Send failed {result}");
}
}
#if CONFIRMED
private static void OnMessageConfirmationHandler()
{
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Send successful");
}
#endif
private static void OnReceiveMessageHandler(byte port, int rssi, int snr, string payload)
{
byte[] payloadBytes = Rak3172LoRaWanDevice.HexToByes(payload); // Done this way so both conversion methods tested
Console.WriteLine($"{DateTime.UtcNow:hh:mm:ss} Receive Message RSSI:{rssi} SNR:{snr} Port:{port} Payload:{payload} PayLoadBytes:{BitConverter.ToString(payloadBytes)}");
}
}
}
I have added XML Documentation comments which will need some rework, after I have “soak tested” the code for at least a week.
I have also added a method so the DevEUI can be set (intended for use after device firmware has been updated), fixed up my mistake with Binary Coded Decimal(BCD) vs. Hexadecimal strings.
I will also go back and apply the “learnings” from this refactoring to my other LoRaWAN module and platform libraries
RaspberyPI OS Software Configuration tool mains screen
RaspberryPI OS IO Serial Port configuration
Raspberry PI OS disabling remote serial login shell
RaspberryPI OS enabling serial port access
Once serial port access was enabled I could enumerate them with SerialPort.GetPortNames() which is in the System.IO.PortsNuGet package. My sample code has compile time options for synchronous and asynchronous operation.
//---------------------------------------------------------------------------------
// Copyright (c) September 2021, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.NetCore.RAK3172.ShieldSerial
{
using System;
using System.Diagnostics;
using System.IO.Ports;
using System.Threading;
public class Program
{
private const string SerialPortId = "/dev/ttyS0";
public static void Main()
{
SerialPort serialPort;
Debug.WriteLine("devMobile.IoT.NetCore.Rak3172.pHatSerial starting");
Debug.WriteLine(String.Join(",", SerialPort.GetPortNames()));
try
{
serialPort = new SerialPort(SerialPortId);
// set parameters
serialPort.BaudRate = 9600;
serialPort.Parity = Parity.None;
serialPort.DataBits = 8;
serialPort.StopBits = StopBits.One;
serialPort.Handshake = Handshake.None;
serialPort.ReadTimeout = 1000;
serialPort.NewLine = "\r\n";
serialPort.Open();
#if SERIAL_ASYNC_READ
serialPort.DataReceived += SerialDevice_DataReceived;
#endif
while (true)
{
serialPort.WriteLine("AT+VER=?");
#if SERIAL_SYNC_READ
// Read the response
string response = serialPort.ReadLine();
Debug.WriteLine($"RX:{response.Trim()} bytes:{response.Length}");
// Read the blank line
response = serialPort.ReadLine();
Debug.WriteLine($"RX:{response.Trim()} bytes:{response.Length}");
// Read the result
response = serialPort.ReadLine();
Debug.WriteLine($"RX:{response.Trim()} bytes:{response.Length}");
#endif
Thread.Sleep(20000);
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
#if SERIAL_ASYNC_READ
private static void SerialDevice_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
SerialPort serialPort = (SerialPort)sender;
switch (e.EventType)
{
case SerialData.Chars:
string response = serialPort.ReadExisting();
Debug.WriteLine($"RX:{response.Trim()} bytes:{response.Length}");
break;
case SerialData.Eof:
Debug.WriteLine("RX :EoF");
break;
default:
Debug.Assert(false, $"e.EventType {e.EventType} unknown");
break;
}
}
#endif
}
}
When I requested the RAK3172 version information with the AT+VER? command the response was three lines, consisting of the version information, a blank line, then the result of the command. If I sent an invalid command the response was two lines, a blank line then “AT_ERROR”
AT+VER? response synchronous
The asynchronous version of the application displays character(s) as they arrive so a response could be split across multiple SerialDataReceived events
AT+VER? response asynchronous
Unlike the RAK811 module the RAK3172 defaults 9600 baud which means there is no need to change the baudrate before using the device. I use the excellent RaspberryDebugger to download application and debug them on my Raspberry PI 3.
devMobile's blog 