Smartish Edge Camera – Azure IoT Tagged Image Upload Error

The SmartEdgeCameraAzureStorageService uploads images with “tags” so it is easier to search for images that may need reviewing. When I added the same tagging functionality to the SmartEdgeCameraAzureIoTService which uploads images to the Storage Account associated with my Azure IoT Hub it failed.

SmartEdgeCameraAzureIoTService error message
[16:39:30.66]fail: devMobile.IoT.MachineLearning.SmartEdgeCameraAzureIoTService.Worker[0]
      Camera image download, post processing, or telemetry failed
      Azure.RequestFailedException: This request is not authorized to perform this operation using this permission.
RequestId:7a1747db-e01e-0019-484c-5c0499000000
Time:2022-04-30T04:39:31.2050951Z
      Status: 403 (This request is not authorized to perform this operation using this permission.)
      ErrorCode: AuthorizationPermissionMismatch

      Content:
      <?xml version="1.0" encoding="utf-8"?><Error><Code>AuthorizationPermissionMismatch</Code><Message>This request is not authorized to perform this operation using this permission.
RequestId:7a1747db-e01e-0019-484c-5c0499000000
Time:2022-04-30T04:39:31.2050951Z</Message></Error>

      Headers:
      Server: Windows-Azure-Blob/1.0,Microsoft-HTTPAPI/2.0
      x-ms-request-id: 7a1747db-e01e-0019-484c-5c0499000000
      x-ms-client-request-id: d0e8eb36-9e01-4eac-a522-f84b9deafa32
      x-ms-version: 2021-04-10
      x-ms-error-code: AuthorizationPermissionMismatch
      Date: Sat, 30 Apr 2022 04:39:30 GMT
      Content-Length: 279
      Content-Type: application/xml

         at Azure.Storage.Blobs.BlockBlobRestClient.UploadAsync(Int64 contentLength, Stream body, Nullable`1 timeout, Byte[] transactionalContentMD5, String blobContentType, String blobContentEncoding, String blobContentLanguage, Byte[] blobContentMD5, String blobCacheControl, IDictionary`2 metadata, String leaseId, String blobContentDisposition, String encryptionKey, String encryptionKeySha256, Nullable`1 encryptionAlgorithm, String encryptionScope, Nullable`1 tier, Nullable`1 ifModifiedSince, Nullable`1 ifUnmodifiedSince, String ifMatch, String ifNoneMatch, String ifTags, String blobTagsString, Nullable`1 immutabilityPolicyExpiry, Nullable`1 immutabilityPolicyMode, Nullable`1 legalHold, CancellationToken cancellationToken)
         at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadInternal(Stream content, BlobHttpHeaders blobHttpHeaders, IDictionary`2 metadata, IDictionary`2 tags, BlobRequestConditions conditions, Nullable`1 accessTier, BlobImmutabilityPolicy immutabilityPolicy, Nullable`1 legalHold, IProgress`1 progressHandler, String operationName, Boolean async, CancellationToken cancellationToken)
         at Azure.Storage.Blobs.Specialized.BlockBlobClient.<>c__DisplayClass62_0.<<GetPartitionedUploaderBehaviors>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
         at Azure.Storage.PartitionedUploader`2.UploadInternal(Stream content, Nullable`1 expectedContentLength, TServiceSpecificData args, IProgress`1 progressHandler, Boolean async, CancellationToken cancellationToken)
         at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadAsync(Stream content, BlobUploadOptions options, CancellationToken cancellationToken)
         at devMobile.IoT.MachineLearning.SmartEdgeCameraAzureIoTService.Worker.UploadImage(List`1 predictions, String filepath, String blobpath) in C:\Users\BrynLewis\source\repos\AzureMLNetSmartEdgeCamera\SmartEdgeCameraAzureIoTService\Worker.cs:line 581
         at devMobile.IoT.MachineLearning.SmartEdgeCameraAzureIoTService.Worker.UploadImage(List`1 predictions, String filepath, String blobpath) in C:\Users\BrynLewis\source\repos\AzureMLNetSmartEdgeCamera\SmartEdgeCameraAzureIoTService\Worker.cs:line 606
         at devMobile.IoT.MachineLearning.SmartEdgeCameraAzureIoTService.Worker.ImageUpdateTimerCallback(Object state) in C:\Users\BrynLewis\source\repos\AzureMLNetSmartEdgeCamera\SmartEdgeCameraAzureIoTService\Worker.cs:line 394
[16:39:30.72]info: devMobile.IoT.MachineLearning.SmartEdgeCameraAzureIoTService.Worker[0]

try
{
   FileUploadSasUriResponse sasUri = await _deviceClient.GetFileUploadSasUriAsync(fileUploadSasUriRequest);

	var blockBlobClient = new BlockBlobClient(sasUri.GetBlobUri());
   ...

   var blockBlobClient = new BlockBlobClient(uploadUri);

	BlobUploadOptions blobUploadOptions = new BlobUploadOptions()
	{
		Tags = new Dictionary<string, string>()
	};

	foreach (var prediction in predictionsTally)
	{
		blobUploadOptions.Tags.Add(prediction.Label, prediction.Count.ToString());
	}   
    await blockBlobClient.UploadAsync(fileStreamSource, blobUploadOptions);
   ...
}
catch (Exception ex)
{
   ...
}

There were no relevant search results(April 2022) so I submitted a Microsoft Azure IoT SDK for .NET issue “UploadAsync fails when Tags added to blob uploading to Storage Account associated with an IoT Hub” which has been triaged and moved to “discussion”.

Smartish Edge Camera – Azure IoT Readonly Properties

This post builds on my Smartish Edge Camera – Azure IoT Direct Methods post adding a number of read only properties. In this version the application reports the OSVersion, MachineName, ApplicationVersion, ImageTimerDue, ImageTimerPeriod, YoloV5ModelPath, PredictionScoreThreshold, PredictionLabelsOfInterest, and PredictionLabelsMinimum.

Azure IoT Explorer displaying the reported “readonly” property values

The AzureMLMetSmartEdgeCamera application supports both Azure IoT Hub and Azure IoT Central connectivity so I have have covered inspecting the properties with Azure IoT Explorer and adding them to an Azure IoT Central Template.

Azure IoT Central Template Readonly properties

The code populates a TwinCollection then calls UpdateReportedPropertiesAsync to push the properties upto my Azure IoT Hub. (This functionality is not available on all Azure IoT hub Tiers)

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
	_logger.LogInformation("Azure IoT Smart Edge Camera Service starting");

	try
	{
#if AZURE_IOT_HUB_CONNECTION
		_deviceClient = await AzureIoTHubConnection();
#endif
#if AZURE_IOT_HUB_DPS_CONNECTION
		_deviceClient = await AzureIoTHubDpsConnection();
#endif

#if AZURE_DEVICE_PROPERTIES
		_logger.LogTrace("ReportedPropeties upload start");

		TwinCollection reportedProperties = new TwinCollection();

		reportedProperties["OSVersion"] = Environment.OSVersion.VersionString;
		reportedProperties["MachineName"] = Environment.MachineName;
		reportedProperties["ApplicationVersion"] = Assembly.GetAssembly(typeof(Program)).GetName().Version;
		reportedProperties["ImageTimerDue"] = _applicationSettings.ImageTimerDue;
		reportedProperties["ImageTimerPeriod"] = _applicationSettings.ImageTimerPeriod;
		reportedProperties["YoloV5ModelPath"] = _applicationSettings.YoloV5ModelPath;

		reportedProperties["PredictionScoreThreshold"] = _applicationSettings.PredictionScoreThreshold;
		reportedProperties["PredictionLabelsOfInterest"] = _applicationSettings.PredictionLabelsOfInterest;
		reportedProperties["PredictionLabelsMinimum"] = _applicationSettings.PredictionLabelsMinimum;

		await _deviceClient.UpdateReportedPropertiesAsync(reportedProperties, stoppingToken);

		_logger.LogTrace("ReportedPropeties upload done");
#endif

		_logger.LogTrace("YoloV5 model setup start");
		_scorer = new YoloScorer<YoloCocoP5Model>(_applicationSettings.YoloV5ModelPath);
		_logger.LogTrace("YoloV5 model setup done");
...

Azure IoT Central Dashboard with readonly properties before UpdateReportedPropertiesAsync called
Azure IoT Central Telemetry displaying property update payloads
Azure IoT Central Dashboard displaying readonly properties

While testing the application I noticed the reported property version was increasing every time I deployed the application. I was retrieving the version information as the application started with AssemblyName.Version

reportedProperties["ApplicationVersion"] = Assembly.GetAssembly(typeof(Program)).GetName().Version;
Visual Studio 2019 Application Package information

I had also configured the Assembly Version in the SmartEdgeCameraAzureIoTService project Package tab to update the assembly build number each time the application was compiled. This was forcing an update of the reported properties version every time the application started

.NET nanoFramework SX127X LoRa library DIO0,DIO1,DIO2,DIO3,DIO4,DIO5

All the previous versions of my .NET nanoFramework Semtech SX127X (LoRa® Mode) library only supported a Dio0 (RegDioMapping1 bits 6&7) EventHandler. This version supports mapping Dio0, Dio1, Dio2, Dio3, Dio4 and Dio5.

DIO Mapping in LoRa Mode
RegDioMapping1 & RegDioMapping2 options

The Dragino Arduino Shield featuring LoRa® technology does not have Dio3 and Dio4 connected so I have been unable to test that functionality.

Dragino LoRa Shield Pin Mapping

The SX127XLoRaDeviceClient main now has OnRxTimeout, OnReceive, OnPayloadCrcError, OnValidHeader, OnTransmit, OnChannelActivityDetectionDone, OnFhssChangeChannel, and OnChannelActivityDetected event handlers (Based on RegIrqFlags bit ordering)

static void Main(string[] args)
{
	int sendCount = 0;
...
#if NETDUINO3_WIFI
	// Arduino D10->PB10
	int chipSelectLine = PinNumber('B', 10);
	// Arduino D9->PE5
	int resetPinNumber = PinNumber('E', 5);
	// Arduino D2 -PA3
	int dio0PinNumber = PinNumber('A', 3);
	// Arduino D6 - PB9
	int dio1PinNumber = PinNumber('B', 9);
	// Arduino D7
	int dio2PinNumber = PinNumber('A', 1);
	// Not connected on Dragino LoRa shield
	//int dio3PinNumber = PinNumber('A', 1);
	//  Not connected on Dragino LoRa shield
	//int dio4PinNumber = PinNumber('A', 1);
	// Arduino D8
	int dio5PinNumber = PinNumber('A', 0);
#endif
...
	Console.WriteLine("devMobile.IoT.SX127xLoRaDevice Client starting");

	try
	{
		var settings = new SpiConnectionSettings(SpiBusId, chipSelectLine)
		{
			ClockFrequency = 1000000,
			Mode = SpiMode.Mode0,// From SemTech docs pg 80 CPOL=0, CPHA=0
			SharingMode = SpiSharingMode.Shared
		};

		using (SpiDevice spiDevice = new SpiDevice(settings))
		using (GpioController gpioController = new GpioController())
		{
...
#if NETDUINO3_WIFI || ST_STM32F769I_DISCOVERY
			sx127XDevice = new SX127XDevice(spiDevice, gpioController, dio0Pin:dio0PinNumber, resetPin:resetPinNumber, dio1Pin: dio1PinNumber, dio2Pin: dio2PinNumber);
#endif

			sx127XDevice.Initialise(Frequency
						, lnaGain: Configuration.RegLnaLnaGain.Default
						, lnaBoost: true
						, powerAmplifier: Configuration.RegPAConfigPASelect.PABoost							
						, rxPayloadCrcOn: true
						, rxDoneignoreIfCrcMissing: false
						);

#if DEBUG
			sx127XDevice.RegisterDump();
#endif

			//sx127XDevice.OnRxTimeout += Sx127XDevice_OnRxTimeout;
			sx127XDevice.OnReceive += SX127XDevice_OnReceive;
			//sx127XDevice.OnPayloadCrcError += Sx127XDevice_OnPayloadCrcError;
			//sx127XDevice.OnValidHeader += Sx127XDevice_OnValidHeader;
			sx127XDevice.OnTransmit += SX127XDevice_OnTransmit;
			//sx127XDevice.OnChannelActivityDetectionDone += Sx127XDevice_OnChannelActivityDetectionDone;
			//sx127XDevice.OnFhssChangeChannel += Sx127XDevice_OnFhssChangeChannel;
			//sx127XDevice.OnChannelActivityDetected += SX127XDevice_OnChannelActivityDetected;

			sx127XDevice.Receive();
			//sx127XDevice.ChannelActivityDetect();

			Thread.Sleep(500);

			while (true)
			{
				string messageText = $"Hello LoRa from .NET nanoFramework Count {sendCount+=1}!";

				byte[] messageBytes = UTF8Encoding.UTF8.GetBytes(messageText);
				Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss}-TX {messageBytes.Length} byte message {messageText}");
				sx127XDevice.Send(messageBytes);

				Thread.Sleep(50000);

				Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Random {sx127XDevice.Random()}");
			}
		}
	}
	catch (Exception ex)
	{
		Console.WriteLine(ex.Message);
	}
}

The Dio0 pin number is the only required pin number parameter, the resetPin, and Dio1 thru Dio5 pin numbers are optional. All the RegDioMapping1 and RegDioMapping2 mappings are disabled on intialisation so there should be no events while the SX127X is being configured.

public SX127XDevice(SpiDevice spiDevice, GpioController gpioController,
	int dio0Pin,
	int resetPin = 0, // Odd order so as not to break exisiting code
	int dio1Pin = 0,
	int dio2Pin = 0,
	int dio3Pin = 0,
	int dio4Pin = 0,
	int dio5Pin = 0
	)
{
	_gpioController = gpioController;

	// Factory reset pin configuration
	if (resetPin != 0)
	{
		_resetPin = resetPin;
		_gpioController.OpenPin(resetPin, PinMode.Output);

		_gpioController.Write(resetPin, PinValue.Low);
		Thread.Sleep(20);
		_gpioController.Write(resetPin, PinValue.High);
		Thread.Sleep(50);
	}

	_registerManager = new RegisterManager(spiDevice, RegisterAddressReadMask, RegisterAddressWriteMask);

	// Once the pins setup check that SX127X chip is present
	Byte regVersionValue = _registerManager.ReadByte((byte)Configuration.Registers.RegVersion);
	if (regVersionValue != Configuration.RegVersionValueExpected)
	{
		throw new ApplicationException("Semtech SX127X not found");
	}

	// See Table 18 DIO Mapping LoRa® Mode
	Configuration.RegDioMapping1 regDioMapping1Value = Configuration.RegDioMapping1.Dio0None;
	regDioMapping1Value |= Configuration.RegDioMapping1.Dio1None;
	regDioMapping1Value |= Configuration.RegDioMapping1.Dio2None;
	regDioMapping1Value |= Configuration.RegDioMapping1.Dio3None;
	_registerManager.WriteByte((byte)Configuration.Registers.RegDioMapping1, (byte)regDioMapping1Value);

	// Currently no easy way to test this with available hardware
	//Configuration.RegDioMapping2 regDioMapping2Value = Configuration.RegDioMapping2.Dio4None;
	//regDioMapping2Value = Configuration.RegDioMapping2.Dio5None;
	//_registerManager.WriteByte((byte)Configuration.Registers.RegDioMapping2, (byte)regDioMapping2Value);

	// Interrupt pin for RXDone, TXDone, and CadDone notification 
	_gpioController.OpenPin(dio0Pin, PinMode.InputPullDown);
	_gpioController.RegisterCallbackForPinValueChangedEvent(dio0Pin, PinEventTypes.Rising, InterruptGpioPin_ValueChanged);

	// RxTimeout, FhssChangeChannel, and CadDetected
	if (dio1Pin != 0)
	{
		_gpioController.OpenPin(dio1Pin, PinMode.InputPullDown);
		_gpioController.RegisterCallbackForPinValueChangedEvent(dio1Pin, PinEventTypes.Rising, InterruptGpioPin_ValueChanged);
	}

	// FhssChangeChannel, FhssChangeChannel, and FhssChangeChannel
	if (dio2Pin != 0)
	{
		_gpioController.OpenPin(dio2Pin, PinMode.InputPullDown);
		_gpioController.RegisterCallbackForPinValueChangedEvent(dio2Pin, PinEventTypes.Rising, InterruptGpioPin_ValueChanged);
	}

	// CadDone, ValidHeader, and PayloadCrcError
	if (dio3Pin != 0)
	{
		_gpioController.OpenPin(dio3Pin, PinMode.InputPullDown);
		_gpioController.RegisterCallbackForPinValueChangedEvent(dio3Pin, PinEventTypes.Rising, InterruptGpioPin_ValueChanged);
	}

	// CadDetected, PllLock and PllLock
	if (dio4Pin != 0)
	{
		_gpioController.OpenPin(dio4Pin, PinMode.InputPullDown);
		_gpioController.RegisterCallbackForPinValueChangedEvent(dio4Pin, PinEventTypes.Rising, InterruptGpioPin_ValueChanged);
	}

	// ModeReady, ClkOut and ClkOut
	if (dio5Pin != 0)
	{
		_gpioController.OpenPin(dio5Pin, PinMode.InputPullDown);
		_gpioController.RegisterCallbackForPinValueChangedEvent(dio5Pin, PinEventTypes.Rising, InterruptGpioPin_ValueChanged);
	}
}

The same event handler (InterruptGpioPin_ValueChanged) is used for Dio0 thru Dio5. Each event has a “process” method and the RegIrqFlags register controls which one(s) are called.

private void InterruptGpioPin_ValueChanged(object sender, PinValueChangedEventArgs pinValueChangedEventArgs)
{
	Byte regIrqFlagsToClear = (byte)Configuration.RegIrqFlags.ClearNone;

	// Read RegIrqFlags to see what caused the interrupt
	Byte irqFlags = _registerManager.ReadByte((byte)Configuration.Registers.RegIrqFlags);

	//Console.WriteLine($"IrqFlags 0x{irqFlags:x} Pin:{pinValueChangedEventArgs.PinNumber}");

	// Check RxTimeout for inbound message
	if ((irqFlags & (byte)Configuration.RegIrqFlagsMask.RxTimeoutMask) == (byte)Configuration.RegIrqFlags.RxTimeout)
	{
		regIrqFlagsToClear |= (byte)Configuration.RegIrqFlags.RxTimeout;

		ProcessRxTimeout(irqFlags);
	}

	// Check RxDone for inbound message
	if ((irqFlags & (byte)Configuration.RegIrqFlagsMask.RxDoneMask) == (byte)Configuration.RegIrqFlags.RxDone)
	{
		regIrqFlagsToClear |= (byte)Configuration.RegIrqFlags.RxDone;

		ProcessRxDone(irqFlags);
	}

	// Check PayLoadCrcError for inbound message
	if ((irqFlags & (byte)Configuration.RegIrqFlagsMask.PayLoadCrcErrorMask) == (byte)Configuration.RegIrqFlags.PayLoadCrcError)
	{
		regIrqFlagsToClear |= (byte)Configuration.RegIrqFlags.PayLoadCrcError;

		ProcessPayloadCrcError(irqFlags);
	}

	// Check ValidHeader for inbound message
	if ((irqFlags & (byte)Configuration.RegIrqFlagsMask.ValidHeaderMask) == (byte)Configuration.RegIrqFlags.ValidHeader)
	{
		regIrqFlagsToClear |= (byte)Configuration.RegIrqFlags.ValidHeader;

		ProcessValidHeader(irqFlags);
	}

	// Check TxDone for outbound message
	if ((irqFlags & (byte)Configuration.RegIrqFlagsMask.TxDoneMask) == (byte)Configuration.RegIrqFlags.TxDone)
	{
		regIrqFlagsToClear |= (byte)Configuration.RegIrqFlags.TxDone;

		ProcessTxDone(irqFlags);
	}

	// Check Channel Activity Detection done 
	if (((irqFlags & (byte)Configuration.RegIrqFlagsMask.CadDoneMask) == (byte)Configuration.RegIrqFlags.CadDone))
	{
		regIrqFlagsToClear |= (byte)Configuration.RegIrqFlags.CadDone;

		ProcessChannelActivityDetectionDone(irqFlags);
	}

	// Check FhssChangeChannel for inbound message
	if ((irqFlags & (byte)Configuration.RegIrqFlagsMask.FhssChangeChannelMask) == (byte)Configuration.RegIrqFlags.FhssChangeChannel)
	{
		regIrqFlagsToClear |= (byte)Configuration.RegIrqFlags.FhssChangeChannel;

		ProcessFhssChangeChannel(irqFlags);
	}

	// Check Channel Activity Detected 
	if (((irqFlags & (byte)Configuration.RegIrqFlagsMask.CadDetectedMask) == (byte)Configuration.RegIrqFlags.CadDetected))
	{
		regIrqFlagsToClear |= (byte)Configuration.RegIrqFlags.CadDetected;

		ProcessChannelActivityDetected(irqFlags);
	}

	_registerManager.WriteByte((byte)Configuration.Registers.RegIrqFlags, regIrqFlagsToClear);
}

private void ProcessRxTimeout(byte irqFlags)
{
	OnRxTimeoutEventArgs onRxTimeoutArgs = new OnRxTimeoutEventArgs();

	OnRxTimeout?.Invoke(this, onRxTimeoutArgs);
}

private void ProcessRxDone(byte irqFlags)
{
	byte[] payloadBytes;
...
}

The RegIrqFlags bits are cleared individually (with regIrqFlagsToClear) at the end of the event handler. Initially I cleared all the flags by writing 0xFF to RegIrqFlags but this caused issues when there were multiple bits set e.g. CadDone along with CadDetected.

devMobile.IoT.SX127xLoRaDevice Client starting
Register dump
Register 0x01 - Value 0X80
...
Register 0x4d - Value 0X84

00:00:09-CAD Detection Done
00:00:09-CAD Detected
00:00:09-RX PacketSnr 0.0 Packet RSSI -100dBm RSSI -96dBm = 9 byte message hello 41
00:00:09-RX PacketSnr 0.0 Packet RSSI -100dBm RSSI -96dBm = 9 byte message hello 42
00:00:09-RX PacketSnr 0.0 Packet RSSI -100dBm RSSI -96dBm = 9 byte message hello 43
00:00:09-RX PacketSnr 0.0 Packet RSSI -100dBm RSSI -96dBm = 9 byte message hello 44
00:00:09-RX PacketSnr 0.0 Packet RSSI -100dBm RSSI -96dBm = 9 byte message hello 45
00:00:09-RX PacketSnr 0.0 Packet RSSI -100dBm RSSI -94dBm = 9 byte message hello 46
00:00:09-RX PacketSnr 0.0 Packet RSSI -99dBm RSSI -94dBm = 9 byte message hello 47
00:00:19-RX PacketSnr 0.0 Packet RSSI -100dBm RSSI -96dBm = 9 byte message hello 48

It took some experimentation with the SX127xLoRaDeviceClient application to “reliably” trigger events for testing. To generate CAD Detected event, I had to modify one of the Arduino-LoRa sample applications to send messages without a delay, then have it running as the SX127xLoRaDeviceClient application was starting.

Smartish Edge Camera – Azure IoT Direct Methods

This post builds on my Smartish Edge Camera – Azure IoT Image-Upload post adding two Direct Methods for Starting and Stopping the image capture and processing timer. The AzureMLMetSmartEdgeCamera supports both Azure IoT Hub and Azure IoT Central connectivity.

Azure IoT Explorer invoking a Direct Method

BEWARE – The Direct Method names are case sensitive which regularly trips me up when I use Azure IoT Explorer. If the Direct Method name is unknown a default handler is called, the issue logged and a Hyper Text Transfer Protocol(HTTP) Not Implemented(501) error returned

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
	_logger.LogInformation("Azure IoT Smart Edge Camera Service starting");

	try
	{
#if AZURE_IOT_HUB_CONNECTION
		_deviceClient = await AzureIoTHubConnection();
#endif
#if AZURE_IOT_HUB_DPS_CONNECTION
		_deviceClient = await AzureIoTHubDpsConnection();
#endif

...
		_logger.LogTrace("YoloV5 model setup start");
		_scorer = new YoloScorer<YoloCocoP5Model>(_applicationSettings.YoloV5ModelPath);
		_logger.LogTrace("YoloV5 model setup done");

		_ImageUpdatetimer = new Timer(ImageUpdateTimerCallback, null, _applicationSettings.ImageTimerDue, _applicationSettings.ImageTimerPeriod);

		await _deviceClient.SetMethodHandlerAsync("ImageTimerStart", ImageTimerStartHandler, null);
		await _deviceClient.SetMethodHandlerAsync("ImageTimerStop", ImageTimerStopHandler, null);
		await _deviceClient.SetMethodDefaultHandlerAsync(DefaultHandler, null);
...
		try
		{
			await Task.Delay(Timeout.Infinite, stoppingToken);
		}
		catch (TaskCanceledException)
		{
			_logger.LogInformation("Application shutown requested");
		}
	}
	catch (Exception ex)
	{
		_logger.LogError(ex, "Application startup failure");
	}
	finally
	{
		_deviceClient?.Dispose();
	}

	_logger.LogInformation("Azure IoT Smart Edge Camera Service shutdown");
}

private async Task<MethodResponse> ImageTimerStartHandler(MethodRequest methodRequest, object userContext)
{
	_logger.LogInformation("ImageUpdatetimer Start Due:{0} Period:{1}", _applicationSettings.ImageTimerDue, _applicationSettings.ImageTimerPeriod);

	_ImageUpdatetimer.Change(_applicationSettings.ImageTimerDue, _applicationSettings.ImageTimerPeriod);

	return new MethodResponse((short)HttpStatusCode.OK);
}

private async Task<MethodResponse> ImageTimerStopHandler(MethodRequest methodRequest, object userContext)
{
	_logger.LogInformation("ImageUpdatetimer Stop");

	_ImageUpdatetimer.Change(Timeout.Infinite, Timeout.Infinite);

	return new MethodResponse((short)HttpStatusCode.OK);
}

private async Task<MethodResponse> DefaultHandler(MethodRequest methodRequest, object userContext)
{
	_logger.LogInformation("Direct Method default handler Name:{0}", methodRequest.Name);

	return new MethodResponse((short)HttpStatusCode.NotFound);
}

I created an Azure IoT Central Template with two command capabilities. (For more detail see my post TTI V3 Connector Azure IoT Central Cloud to Device(C2D)).

Azure IoT Central Template Direct Method configuration
Azure IoT Central Template Direct Method invocation
Azure Smart Edge Camera console application Start Direct Method call

Initially, I had one long post which covered Direct Methods, Readonly Properties and Updateable Properties but it got too long so I split it into three.

.NET nanoFramework SX127X LoRa library RegPaConfig RegPaDac

While updating my .NET nanoFramework Semtech SX127X library I revisited (because I thought it might still be wrong) how the output power is calculated. I started with the overview of the transmitter architecture in in the datasheet…

SX127X Overview of transmission pipeline

The RegPaConfig register has three settings PaSelect(RFO & PA_BOOST), MaxPower(0..7), and OutputPower(0..15). When in RFO mode the pOut has a range of -4 to 15 and PA_BOOST mode has a range of 2 to 20. (The AdaFruit version of the RadioHead library has differences to the Semtech Lora-net/LoRaMac-Node libraries)

RegPaConfig & RegOcp register configuration options

The SX127X also has a power amplifier attached to the PA_BOOST pin and a higher power amplifier which is controlled by the RegPaDac register.

High power mode overview
RegPaDac register configuration options

The RegOcp (over current protection) has to be relaxed for the higher power modes

RegPaConfig register configuration options

I started with the Semtech Lora-net/LoRaMac-Node library which reads the RegPaConfig, RegPaSelect and RegPaDac registers then does any updates required.

void SX1276SetRfTxPower( int8_t power )
{
    uint8_t paConfig = 0;
    uint8_t paDac = 0;

    paConfig = SX1276Read( REG_PACONFIG );
    paDac = SX1276Read( REG_PADAC );

    paConfig = ( paConfig & RF_PACONFIG_PASELECT_MASK ) | SX1276GetPaSelect( power );

    if( ( paConfig & RF_PACONFIG_PASELECT_PABOOST ) == RF_PACONFIG_PASELECT_PABOOST )
    {
        if( power > 17 )
        {
            paDac = ( paDac & RF_PADAC_20DBM_MASK ) | RF_PADAC_20DBM_ON;
        }
        else
        {
            paDac = ( paDac & RF_PADAC_20DBM_MASK ) | RF_PADAC_20DBM_OFF;
        }
        if( ( paDac & RF_PADAC_20DBM_ON ) == RF_PADAC_20DBM_ON )
        {
            if( power < 5 )
            {
                power = 5;
            }
            if( power > 20 )
            {
                power = 20;
            }
            paConfig = ( paConfig & RF_PACONFIG_OUTPUTPOWER_MASK ) | ( uint8_t )( ( uint16_t )( power - 5 ) & 0x0F );
        }
        else
        {
            if( power < 2 )
            {
                power = 2;
            }
            if( power > 17 )
            {
                power = 17;
            }
            paConfig = ( paConfig & RF_PACONFIG_OUTPUTPOWER_MASK ) | ( uint8_t )( ( uint16_t )( power - 2 ) & 0x0F );
        }
    }
    else
    {
        if( power > 0 )
        {
            if( power > 15 )
            {
                power = 15;
            }
            paConfig = ( paConfig & RF_PACONFIG_MAX_POWER_MASK & RF_PACONFIG_OUTPUTPOWER_MASK ) | ( 7 << 4 ) | ( power );
        }
        else
        {
            if( power < -4 )
            {
                power = -4;
            }
            paConfig = ( paConfig & RF_PACONFIG_MAX_POWER_MASK & RF_PACONFIG_OUTPUTPOWER_MASK ) | ( 0 << 4 ) | ( power + 4 );
        }
    }
    SX1276Write( REG_PACONFIG, paConfig );
    SX1276Write( REG_PADAC, paDac );
}

I also reviewed the Arduino-LoRa Semtech library which only writes to the RegPaConfig, RegPaSelect and RegPaDac registers.

void LoRaClass::setTxPower(int level, int outputPin)
{
  if (PA_OUTPUT_RFO_PIN == outputPin) {
    // RFO
    if (level < 0) {
      level = 0;
    } else if (level > 14) {
      level = 14;
    }

    writeRegister(REG_PA_CONFIG, 0x70 | level);
  } else {
    // PA BOOST
    if (level > 17) {
      if (level > 20) {
        level = 20;
      }

      // subtract 3 from level, so 18 - 20 maps to 15 - 17
      level -= 3;

      // High Power +20 dBm Operation (Semtech SX1276/77/78/79 5.4.3.)
      writeRegister(REG_PA_DAC, 0x87);
      setOCP(140);
    } else {
      if (level < 2) {
        level = 2;
      }
      //Default value PA_HF/LF or +17dBm
      writeRegister(REG_PA_DAC, 0x84);
      setOCP(100);
    }

    writeRegister(REG_PA_CONFIG, PA_BOOST | (level - 2));
  }
}

I updated the output power configuration code in the Initialise method of the SX127X library. After reviewing the SX127X datasheet I extended the way the pOut is calculated in RFO mode. The code uses two values for MaxPower(RegPAConfigMaxPower.Min & RegPAConfigMaxPower.Max) so that the full RTO output power range was available.

// Set RegPAConfig & RegPaDac if powerAmplifier/OutputPower settings not defaults
if ((powerAmplifier != Configuration.RegPAConfigPASelect.Default) || (outputPower != Configuration.OutputPowerDefault))
{
	if (powerAmplifier == Configuration.RegPAConfigPASelect.PABoost)
	{
		byte regPAConfigValue = (byte)Configuration.RegPAConfigPASelect.PABoost;

		// Validate the minimum and maximum PABoost outputpower
		if ((outputPower < Configuration.OutputPowerPABoostMin) || (outputPower > Configuration.OutputPowerPABoostMax))
		{
			throw new ApplicationException($"PABoost {outputPower}dBm Min power {Configuration.OutputPowerPABoostMin} to Max power {Configuration.OutputPowerPABoostMax}");
		}

		if (outputPower < Configuration.OutputPowerPABoostPaDacThreshhold)
		{
			// outputPower 0..15 so pOut is 2=17-(15-0)...17=17-(15-15)
			regPAConfigValue |= (byte)Configuration.RegPAConfigMaxPower.Default;
			regPAConfigValue |= (byte)(outputPower - 2);

			_registerManager.WriteByte((byte)Configuration.Registers.RegPAConfig, regPAConfigValue);
			_registerManager.WriteByte((byte)Configuration.Registers.RegPaDac, (byte)Configuration.RegPaDac.Normal);
		}
		else
		{
			// outputPower 0..15 so pOut is 5=20-(15-0)...20=20-(15-15) // See https://github.com/adafruit/RadioHead/blob/master/RH_RF95.cpp around line 411 could be 23dBm
			regPAConfigValue |= (byte)Configuration.RegPAConfigMaxPower.Default;
			regPAConfigValue |= (byte)(outputPower - 5);

			_registerManager.WriteByte((byte)Configuration.Registers.RegPAConfig, regPAConfigValue);
			_registerManager.WriteByte((byte)Configuration.Registers.RegPaDac, (byte)Configuration.RegPaDac.Boost);
		}
	}
	else
	{
		byte regPAConfigValue = (byte)Configuration.RegPAConfigPASelect.Rfo;

		// Validate the minimum and maximum RFO outputPower
		if ((outputPower < Configuration.OutputPowerRfoMin) || (outputPower > Configuration.OutputPowerRfoMax))
		{
			throw new ApplicationException($"RFO {outputPower}dBm Min power {Configuration.OutputPowerRfoMin} to Max power {Configuration.OutputPowerRfoMax}");
		}

		// Set MaxPower and Power calculate pOut = PMax-(15-outputPower), pMax=10.8 + 0.6*MaxPower 
		if (outputPower > Configuration.OutputPowerRfoThreshhold)
		{
			// pMax 15=10.8+0.6*7 with outputPower 0...15 so pOut is 15=pMax-(15-0)...0=pMax-(15-15) 
			regPAConfigValue |= (byte)Configuration.RegPAConfigMaxPower.Max;
			regPAConfigValue |= (byte)(outputPower + 0);
		}
		else
		{
			// pMax 10.8=10.8+0.6*0 with output power 0..15 so pOut is -4=10-(15-0)...10.8=10.8-(15-15)
			 regPAConfigValue |= (byte)Configuration.RegPAConfigMaxPower.Min;
			regPAConfigValue |= (byte)(outputPower + 4);
		}

		_registerManager.WriteByte((byte)Configuration.Registers.RegPAConfig, regPAConfigValue);
		_registerManager.WriteByte((byte)Configuration.Registers.RegPaDac, (byte)Configuration.RegPaDac.Normal);
	}
}

The formula for pOut and pMax in RegPaConfig documentation is included in the source code so I could manually calculate (including edge cases) the values as part of my testing. I ran the SX127XLoRaDeviceClient and inspected the PaConfig & RegPaDac in the Visual Studio 2022 debugger.

PABoost
Output power = 1
Output power = 21
Exception

Output power = 2
PaConfig = 192
RegPaDac = normal
	1100 0000

Output power = 16
PaConfig = 206
RegPaDac = normal
	1100 1110

Output power = 17
PaConfig = 204
RegPacDac = Normal
	1100 1100

Output power = 18
PaConfig = 205
RegPacDac = Boost
	1100 1101

Output power = 19
PaConfig = 206
RegPacDac = Boost
	1100 1110

Output power = 20
PaConfig = 207
RegPacDac = Boost
	1100 1111

RFO
Output power = -5
Output power = 16
Exception

Output power = -4
PAConfig = 0
	0000 0000

Output power = -1
PAConfig = 3
	0000 0011

Output power = 0
PAConfig = 4
	0000 0100

Output power = 1
PAConfig = 113
	0111 0001

OutputPower = 14
PAConfig = 126
	0111 1110

OutputPower = 15
PAConfig = 127
	0111 1111

I need to borrow some test gear to check my implementation

Smartish Edge Camera – Azure IoT Image Upload

This post builds on my Smartish Edge Camera – Azure Storage Service, Azure IoT Hub, and Azure IoT Central projects adding optional camera and marked-up image upload to Azure Blob Storage for Azure IoT Hubs and Azure IoT Central.

Azure IoT Hub – File upload storage account configuration
Azure IoT Central – File upload storage account configuration

The “new improved” process of uploading files to an Azure IoT Hub and Azure IoT Central is surprisingly complex to use and make robust(I think the initial approach with DeviceClient.UploadToBlobAsync which is now “deprecated” was easier to use).

public async Task UploadImage(List<YoloPrediction> predictions, string filepath, string blobpath)
{
	var fileUploadSasUriRequest = new FileUploadSasUriRequest()
	{
		BlobName = blobpath 
	};

	FileUploadSasUriResponse sasUri = await _deviceClient.GetFileUploadSasUriAsync(fileUploadSasUriRequest);

	var blockBlobClient = new BlockBlobClient(sasUri.GetBlobUri());

	var fileUploadCompletionNotification = new FileUploadCompletionNotification()
	{
		// Mandatory. Must be the same value as the correlation id returned in the sas uri response
		CorrelationId = sasUri.CorrelationId,

		IsSuccess = true
	};

	try
	{
		using (FileStream fileStream = File.OpenRead(filepath))
		{
			Response<BlobContentInfo> response = await blockBlobClient.UploadAsync(fileStream); //, blobUploadOptions);

			fileUploadCompletionNotification.StatusCode = response.GetRawResponse().Status;

			if (fileUploadCompletionNotification.StatusCode != ((int)HttpStatusCode.Created))
			{
				fileUploadCompletionNotification.IsSuccess = false;

				fileUploadCompletionNotification.StatusDescription = response.GetRawResponse().ReasonPhrase;
			}
		}
	}
	catch (RequestFailedException ex)
	{
		fileUploadCompletionNotification.StatusCode = ex.Status;

		fileUploadCompletionNotification.IsSuccess = false;

		fileUploadCompletionNotification.StatusDescription = ex.Message;
	}
	finally
	{
		await _deviceClient.CompleteFileUploadAsync(fileUploadCompletionNotification);
	}
}

If there is an object with a label in the PredictionLabelsOfInterest list, the camera and marked-up images can (configured with ImageCameraUpload & ImageMarkedupUpload) be uploaded to an Azure Storage Blob container associated with an Azure IoT Hub/ Azure IoT Central instance.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },

  "Application": {
    "DeviceID": "",
    "ImageTimerDue": "0.00:00:15",
    "ImageTimerPeriod": "0.00:00:30",

    "ImageCameraFilepath": "ImageCamera.jpg",
    "ImageMarkedUpFilepath": "ImageMarkedup.jpg",

    "ImageCameraUpload": false,
    "ImageMarkedupUpload": true,

    "ImageUploadFilepath": "ImageMarkedup.jpg",

    "YoloV5ModelPath": "YoloV5/yolov5s.onnx",

    "PredictionScoreThreshold": 0.7,
    "PredictionLabelsOfInterest": [
      "bicycle",
      "person"
    ],

    "PredictionLabelsMinimum": [
      "bicycle",
      "car",
      "person"
    ],

    "ImageCameraFilenameFormat": "{0:yyyyMMdd}/{0:HHmmss}.jpg"
  },

  "SecurityCamera": {
    "CameraUrl": "",
    "CameraUserName": "",
    "CameraUserPassword": ""
  },

  "RaspberryPICamera": {
    "ProcessWaitForExit": 1000,
    "Rotation": 180
  },

  "AzureIoTHub": {
    "ConnectionString": ""
  },

  "AzureIoTHubDPS": {
    "GlobalDeviceEndpoint": "global.azure-devices-provisioning.net",
    "IDScope": "",
    "GroupEnrollmentKey": ""
  },

  "AzureStorage": {
    "ImageCameraFilenameFormat": "{0:yyyyMMdd}/camera/{0:HHmmss}.jpg",
    "ImageMarkedUpFilenameFormat": "{0:yyyyMMdd}/markedup/{0:HHmmss}.jpg"
  }
}

The Blob’s path is prefixed with the device id (My Azure Storage Service created an Azure Blob Storage container for each device).

Azure IoT Central SmartEdge Camera devices

The format of the Azure Storage Blob path is configurable(ImageCameraFilenameFormat & ImageMarkedUpFilenameFormat + Universal Coordinated Time(UTC)) so images can be grouped.

Configurable Blob paths in Azure Storage Explorer

After creating a new Azure IoT Hub uploads started failing with an exception and there weren’t a lot of useful search results (April 2022). I found error this was caused by missing or incorrect Azure Storage Account configuration.

Azure IoT Hub Upload application failure logging
{"Message":"{\"errorCode\":400022,\"trackingId\":\"1175af36ec884cc4a54978f77b877a01-G:0-TimeStamp:04/12/2022 10:19:04\",\"message\":\"BadRequest\",\"timestampUtc\":\"2022-04-12T10:19:04.5925999Z\"}","ExceptionMessage":""}

   at Microsoft.Azure.Devices.Client.Transport.HttpClientHelper.<ExecuteAsync>d__23.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Devices.Client.Transport.HttpClientHelper.<PostAsync>d__19`2.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Azure.Devices.Client.Transport.HttpTransportHandler.<GetFileUploadSasUriAsync>d__15.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at devMobile.IoT.MachineLearning.SmartEdgeCameraAzureIoTService.Worker.<UploadImage>d__14.MoveNext() in C:\Users\BrynLewis\source\repos\AzureMLNetSmartEdgeCamera\SmartEdgeCameraAzureIoTService\Worker.cs:line 430
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at devMobile.IoT.MachineLearning.SmartEdgeCameraAzureIoTService.Worker.<ImageUpdateTimerCallback>d__10.MoveNext() in C:\Users\BrynLewis\source\repos\AzureMLNetSmartEdgeCamera\SmartEdgeCameraAzureIoTService\Worker.cs:line 268

While testing the application I noticed an “unexpected” object detected in my backyard…

Unexpected object detection diagnostic logging
Unexpected object detection results marked-up image

The mentalstack/yolov5-net and NuGet have been incredibly useful and MentalStack team have done a marvelous job building and supporting this project. For this project my test-rig consisted of a Unv ADZK-10 Security Camera, Power over Ethernet(PoE) and my HP Prodesk 400G4 DM (i7-8700T).

Smartish Edge Camera – Azure IoT Central

This post builds on Smartish Edge Camera – Azure Hub Part 1 using the Azure IoT Hub Device Provisioning Service(DPS) to connect to Azure IoT Central.

The list of object classes is in the YoloCocoP5Model.cs file in the mentalstack/yolov5-net repository.

public override List<YoloLabel> Labels { get; set; } = new List<YoloLabel>()
{
    new YoloLabel { Id = 1, Name = "person" },
    new YoloLabel { Id = 2, Name = "bicycle" },
    new YoloLabel { Id = 3, Name = "car" },
    new YoloLabel { Id = 4, Name = "motorcycle" },
    new YoloLabel { Id = 5, Name = "airplane" },
    new YoloLabel { Id = 6, Name = "bus" },
    new YoloLabel { Id = 7, Name = "train" },
    new YoloLabel { Id = 8, Name = "truck" },
    new YoloLabel { Id = 9, Name = "boat" },
    new YoloLabel { Id = 10, Name = "traffic light" },
    new YoloLabel { Id = 11, Name = "fire hydrant" },
    new YoloLabel { Id = 12, Name = "stop sign" },
    new YoloLabel { Id = 13, Name = "parking meter" },
    new YoloLabel { Id = 14, Name = "bench" },
    new YoloLabel { Id = 15, Name = "bird" },
    new YoloLabel { Id = 16, Name = "cat" },
    new YoloLabel { Id = 17, Name = "dog" },
    new YoloLabel { Id = 18, Name = "horse" },
    new YoloLabel { Id = 19, Name = "sheep" },
    new YoloLabel { Id = 20, Name = "cow" },
    new YoloLabel { Id = 21, Name = "elephant" },
    new YoloLabel { Id = 22, Name = "bear" },
    new YoloLabel { Id = 23, Name = "zebra" },
    new YoloLabel { Id = 24, Name = "giraffe" },
    new YoloLabel { Id = 25, Name = "backpack" },
    new YoloLabel { Id = 26, Name = "umbrella" },
    new YoloLabel { Id = 27, Name = "handbag" },
    new YoloLabel { Id = 28, Name = "tie" },
    new YoloLabel { Id = 29, Name = "suitcase" },
    new YoloLabel { Id = 30, Name = "frisbee" },
    new YoloLabel { Id = 31, Name = "skis" },
    new YoloLabel { Id = 32, Name = "snowboard" },
    new YoloLabel { Id = 33, Name = "sports ball" },
    new YoloLabel { Id = 34, Name = "kite" },
    new YoloLabel { Id = 35, Name = "baseball bat" },
    new YoloLabel { Id = 36, Name = "baseball glove" },
    new YoloLabel { Id = 37, Name = "skateboard" },
    new YoloLabel { Id = 38, Name = "surfboard" },
    new YoloLabel { Id = 39, Name = "tennis racket" },
    new YoloLabel { Id = 40, Name = "bottle" },
    new YoloLabel { Id = 41, Name = "wine glass" },
    new YoloLabel { Id = 42, Name = "cup" },
    new YoloLabel { Id = 43, Name = "fork" },
    new YoloLabel { Id = 44, Name = "knife" },
    new YoloLabel { Id = 45, Name = "spoon" },
    new YoloLabel { Id = 46, Name = "bowl" },
    new YoloLabel { Id = 47, Name = "banana" },
    new YoloLabel { Id = 48, Name = "apple" },
    new YoloLabel { Id = 49, Name = "sandwich" },
    new YoloLabel { Id = 50, Name = "orange" },
    new YoloLabel { Id = 51, Name = "broccoli" },
    new YoloLabel { Id = 52, Name = "carrot" },
    new YoloLabel { Id = 53, Name = "hot dog" },
    new YoloLabel { Id = 54, Name = "pizza" },
    new YoloLabel { Id = 55, Name = "donut" },
    new YoloLabel { Id = 56, Name = "cake" },
    new YoloLabel { Id = 57, Name = "chair" },
    new YoloLabel { Id = 58, Name = "couch" },
    new YoloLabel { Id = 59, Name = "potted plant" },
    new YoloLabel { Id = 60, Name = "bed" },
    new YoloLabel { Id = 61, Name = "dining table" },
    new YoloLabel { Id = 62, Name = "toilet" },
    new YoloLabel { Id = 63, Name = "tv" },
    new YoloLabel { Id = 64, Name = "laptop" },
    new YoloLabel { Id = 65, Name = "mouse" },
    new YoloLabel { Id = 66, Name = "remote" },
    new YoloLabel { Id = 67, Name = "keyboard" },
    new YoloLabel { Id = 68, Name = "cell phone" },
    new YoloLabel { Id = 69, Name = "microwave" },
    new YoloLabel { Id = 70, Name = "oven" },
    new YoloLabel { Id = 71, Name = "toaster" },
    new YoloLabel { Id = 72, Name = "sink" },
    new YoloLabel { Id = 73, Name = "refrigerator" },
    new YoloLabel { Id = 74, Name = "book" },
    new YoloLabel { Id = 75, Name = "clock" },
    new YoloLabel { Id = 76, Name = "vase" },
    new YoloLabel { Id = 77, Name = "scissors" },
    new YoloLabel { Id = 78, Name = "teddy bear" },
    new YoloLabel { Id = 79, Name = "hair drier" },
    new YoloLabel { Id = 80, Name = "toothbrush" }
};

Some of the label choices seem a bit arbitrary(frisbee, surfboard) and American(fire hydrant, baseball bat, baseball glove) It was quite tedious configuring the 80 labels in my Azure IoT Central template.

Azure IoT Central Template with all the YoloV5 labels configured

If there is an object with a label in the PredictionLabelsOfInterest list, a tally of each of the different object classes in the image is sent to an Azure IoT Hub/ Azure IoT Central.

"Application": {
  "DeviceID": "",
  "ImageTimerDue": "0.00:00:15",
  "ImageTimerPeriod": "0.00:00:30",

  "ImageCameraFilepath": "ImageCamera.jpg",
  "ImageMarkedUpFilepath": "ImageMarkedup.jpg",

  "YoloV5ModelPath": "YoloV5/yolov5s.onnx",

  "PredictionScoreThreshold": 0.7,
  "PredictionLabelsOfInterest": [
    "bicycle",
    "person"
  ],
  "PredictionLabelsMinimum": [
    "bicycle",
    "car",
    "person"
  ]
}
My backyard just after the car left (the dry patch in shingle on the right)
Smartish Edge Camera Service console just after car left
Smartish Edge Camera Azure IoT Central graphs showing missing data points

After the You Only Look Once(YOLOV5)+ML.Net+Open Neural Network Exchange(ONNX) plumbing has loaded a timer with a configurable due time and period is started.

private async void ImageUpdateTimerCallback(object state)
{
	DateTime requestAtUtc = DateTime.UtcNow;

	// Just incase - stop code being called while photo already in progress
	if (_cameraBusy)
	{
		return;
	}
	_cameraBusy = true;

	_logger.LogInformation("Image processing start");

	try
	{
#if CAMERA_RASPBERRY_PI
		RaspberryPIImageCapture();
#endif
#if CAMERA_SECURITY
		SecurityCameraImageCapture();
#endif
		List<YoloPrediction> predictions;

		using (Image image = Image.FromFile(_applicationSettings.ImageCameraFilepath))
		{
			_logger.LogTrace("Prediction start");
			predictions = _scorer.Predict(image);
			_logger.LogTrace("Prediction done");

			OutputImageMarkup(image, predictions, _applicationSettings.ImageMarkedUpFilepath);
		}

		if (_logger.IsEnabled(LogLevel.Trace))
		{
			_logger.LogTrace("Predictions {0}", predictions.Select(p => new { p.Label.Name, p.Score }));
		}

		var predictionsValid = predictions.Where(p => p.Score >= _applicationSettings.PredictionScoreThreshold).Select(p => p.Label.Name);

		// Count up the number of each class detected in the image
		var predictionsTally = predictionsValid.GroupBy(p => p)
				.Select(p => new
				{
					Label = p.Key,
					Count = p.Count()
				});

		if (_logger.IsEnabled(LogLevel.Information))
		{
			_logger.LogInformation("Predictions tally before {0}", predictionsTally.ToList());
		}

		// Add in any missing counts the cloudy side is expecting
		if (_applicationSettings.PredictionLabelsMinimum != null)
		{
			foreach( String label in _applicationSettings.PredictionLabelsMinimum)
			{
				if (!predictionsTally.Any(c=>c.Label == label ))
				{
					predictionsTally = predictionsTally.Append(new {Label = label, Count = 0 });
				}
			}
		}

		if (_logger.IsEnabled(LogLevel.Information))
		{
			_logger.LogInformation("Predictions tally after {0}", predictionsTally.ToList());
		}

		if ((_applicationSettings.PredictionLabelsOfInterest == null) || (predictionsValid.Select(c => c).Intersect(_applicationSettings.PredictionLabelsOfInterest, StringComparer.OrdinalIgnoreCase).Any()))
		{
			JObject telemetryDataPoint = new JObject();

			foreach (var predictionTally in predictionsTally)
			{
				telemetryDataPoint.Add(predictionTally.Label, predictionTally.Count);
			}

			using (Message message = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryDataPoint))))
			{
				message.Properties.Add("iothub-creation-time-utc", requestAtUtc.ToString("s", CultureInfo.InvariantCulture));

				await _deviceClient.SendEventAsync(message);
			}
		}
	}
	catch (Exception ex)
	{
		_logger.LogError(ex, "Camera image download, post processing, or telemetry failed");
	}
	finally
	{
		_cameraBusy = false;
	}

	TimeSpan duration = DateTime.UtcNow - requestAtUtc;

	_logger.LogInformation("Image processing done {0:f2} sec", duration.TotalSeconds);
}

Using some Language Integrated Query (LINQ) code any predictions with a score < PredictionScoreThreshold are discarded. A count of the instances of each class is generated with some more LINQ code.

The PredictionLabelsMinimum(optional) is then used to add additional labels with a count of 0 to PredictionsTally so there are no missing datapoints. This is specifically for Azure IoT Central Dashboard so the graph lines are continuous.

Smartish Edge Camera Service console just after put bike in-front of the garage

If any of the list of valid predictions labels is in the PredictionLabelsOfInterest list (if the PredictionLabelsOfInterest is empty any label is a label of interest) the list of prediction class counts is used to populate a Newtonsoft JObject which is serialised to generate a Java Script Object Notation(JSON) Azure IoT Hub message payload.

The “automagic” graph scaling can be sub-optimal

The mentalstack/yolov5-net and NuGet have been incredibly useful and MentalStack team have done a marvelous job building and supporting this project.

The test-rig consisted of a Unv ADZK-10 Security Camera, Power over Ethernet(PoE) and my HP Prodesk 400G4 DM (i7-8700T).

Smartish Edge Camera – Azure IoT Hub

The SmartEdgeCameraAzureIoTService application uses the same You Only Look Once(YOLOV5) + ML.Net + Open Neural Network Exchange(ONNX) plumbing as the SmartEdgeCameraAzureStorageService.

If there is an object with a label in the PredictionLabelsOfInterest list, a tally of each of the different object classes is sent to an Azure IoT Hub.

"Application": {
  "DeviceID": "",
  "ImageTimerDue": "0.00:00:15",
  "ImageTimerPeriod": "0.00:00:30",

  "ImageCameraFilepath": "ImageCamera.jpg",

  "YoloV5ModelPath": "YoloV5/yolov5s.onnx",

  "PredicitionScoreThreshold": 0.7,
  "PredictionLabelsOfInterest": [
    "person"
  ],
}

The Azure IoT hub can configured via a Shared Access Signature(SAS) device policy connection string or the Azure IoT Hub Device Provisioning Service(DPS)

Cars and bicycles in my backyard with no object(s) of interest
SmartEdgeCameraAzureIoTService no object(s) of interest
Cars and bicycles in my backyard with one object of interest
SmartEdgeCameraAzureIoTService one object of interest
Azure IoT Explorer Telemetry with one object of interest

After the You Only Look Once(YOLOV5)+ML.Net+Open Neural Network Exchange(ONNX) plumbing has loaded a timer with a configurable due time and period is started. Using some Language Integrated Query (LINQ) code any predictions with a score < PredictionScoreThreshold are discarded, then the list of predictions is checked to see if there are any in the PredictionLabelsOfInterest. If there are any matching predictions a count of the instances of each class is generated with more LINQ code.

private async void ImageUpdateTimerCallback(object state)
{
	DateTime requestAtUtc = DateTime.UtcNow;

	// Just incase - stop code being called while photo already in progress
	if (_cameraBusy)
	{
		return;
	}
	_cameraBusy = true;

	_logger.LogInformation("Image processing start");

	try
	{
#if CAMERA_RASPBERRY_PI
		RaspberryPIImageCapture();
#endif
#if CAMERA_SECURITY
		SecurityCameraImageCapture();
#endif
		List<YoloPrediction> predictions;

		using (Image image = Image.FromFile(_applicationSettings.ImageCameraFilepath))
		{
			_logger.LogTrace("Prediction start");
			predictions = _scorer.Predict(image);
			_logger.LogTrace("Prediction done");
		}

		if (_logger.IsEnabled(LogLevel.Trace))
		{
			_logger.LogTrace("Predictions {0}", predictions.Select(p => new { p.Label.Name, p.Score }));
		}

		var predictionsOfInterest = predictions.Where(p => p.Score > _applicationSettings.PredicitionScoreThreshold)
										.Select(c => c.Label.Name)
										.Intersect(_applicationSettings.PredictionLabelsOfInterest, StringComparer.OrdinalIgnoreCase);

		if (predictionsOfInterest.Any())
		{
			if (_logger.IsEnabled(LogLevel.Trace))
			{
				_logger.LogTrace("Predictions of interest {0}", predictionsOfInterest.ToList());
			}

			var predictionsTally = predictions.GroupBy(p => p.Label.Name)
									.Select(p => new
									{
										Label = p.Key,
										Count = p.Count()
									});

			if (_logger.IsEnabled(LogLevel.Information))
			{
				_logger.LogInformation("Predictions tally {0}", predictionsTally.ToList());
			}

			JObject telemetryDataPoint = new JObject();

			foreach (var predictionTally in predictionsTally)
			{
				telemetryDataPoint.Add(predictionTally.Label, predictionTally.Count);
			}

			using (Message message = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryDataPoint))))
			{
				message.Properties.Add("iothub-creation-time-utc", requestAtUtc.ToString("s", CultureInfo.InvariantCulture));

				await _deviceClient.SendEventAsync(message);
			}
		}
	}
	catch (Exception ex)
	{
		_logger.LogError(ex, "Camera image download, post processing, telemetry failed");
	}
	finally
	{
		_cameraBusy = false;
	}

	TimeSpan duration = DateTime.UtcNow - requestAtUtc;

	_logger.LogInformation("Image processing done {0:f2} sec", duration.TotalSeconds);
}

The list of prediction class counts is used to populate a Newtonsoft JObject which serialised to generate a Java Script Object Notation(JSON) payload for an Azure IoT Hub message.

The test-rig consisted of a Unv ADZK-10 Security Camera, Power over Ethernet(PoE) and my HP Prodesk 400G4 DM (i7-8700T)

.NET nanoFramework SX127X LoRa library RegLna LnaGain

Every so often I print my code out (landscape for notes in margin, double sided to save paper, and colour so it looks like Visual Studio 2022) and within 100 lines noticed the first of no doubt many issues. The SX127X RegLNA enumeration was wrong.

// RegLna
[Flags]
public enum RegLnaLnaGain : byte
{
	G1 = 0b00000001,
	G2 = 0b00000010,
	G3 = 0b00000011,
	G4 = 0b00000100,
	G5 = 0b00000101,
	G6 = 0b00000110
}
SX127X RegLna options

The LnaGain value is bits 5-7 rather than rather than bits 0-2 which could be a problem if the specified lnaGain and lnaBoost values are not the default values.

// Set RegLna if any of the settings not defaults
if ((lnaGain != Configuration.LnaGainDefault) || (lnaBoost != Configuration.LnaBoostDefault))
{
	byte regLnaValue = (byte)lnaGain;

	regLnaValue |= Configuration.RegLnaLnaBoostLfDefault;
	regLnaValue |= Configuration.RegLnaLnaBoostHfDefault;

	if (lnaBoost)
	{
		if (_frequency > Configuration.SX127XMidBandThreshold)
		{
			regLnaValue |= Configuration.RegLnaLnaBoostHfOn;
		}
		else
		{
			regLnaValue |= Configuration.RegLnaLnaBoostLfOn;
		}
	}
	_registerManager.WriteByte((byte)Configuration.Registers.RegLna, regLnaValue);
}

The default lnaGain is G1 and the default lnaBoost is false so if the gain was set to G3(011) then LnaBoostHf current would be 150% and LnaGain would be 000 which is a reserved value.

// RegLna
[Flags]
public enum RegLnaLnaGain : byte
{
	G1 = 0b00100000,
	G2 = 0b01000000,
	G3 = 0b01100000,
	G4 = 0b10000000,
	G5 = 0b10100000,
	G6 = 0b11000000
}

I need to check my usage of Configuration.SX127XMidBandThreshold for LnaBoostLf vs. LnaBoostHf is correct.(arduino-LoRa)