Random wanderings through Microsoft Azure esp. PaaS plumbing, the IoT bits, AI on Micro controllers, AI on Edge Devices, .NET nanoFramework, .NET Core on *nix and ML.NET+ONNX
Still couldn’t figure out why my code was failing so I turned up logging to 11 and noticed a couple of messages which didn’t make sense. The device was connecting than disconnecting which indicated a another problem. As part of the Message Queue Telemetry Transport(MQTT) specification there is a “feature” Last Will and Testament(LWT) which a client can configure so that the MQTT broker sends a message to a topic if the device disconnects unexpectedly.
I was looking at the code and noticed that LWT was being used and that the topic didn’t exist in my Azure Event Grid MQTT Broker namespace. When the LWT configuration was commented out the application worked.
Paying close attention to the logging I noticed the “Subscribing to ssl/mqtts” followed by “Subscribe request sent”
I checked the sample application and found that if the connect was successful the application would then try and subscribe to a topic that didn’t exist.
The PEM encoded root CA certificate chain that is used to validate the server
public const string CA_ROOT_PEM = @"-----BEGIN CERTIFICATE-----
CN: CN = Microsoft Azure ECC TLS Issuing CA 03
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
CN: CN = DigiCert Global Root G3
-----END CERTIFICATE-----";
The PEM encoded certificate chain that is used to authenticate the device
public const string CLIENT_CERT_PEM_A = @"-----BEGIN CERTIFICATE-----
-----BEGIN CERTIFICATE-----
CN=Self signed device certificate
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
CN=Self signed Intermediate certificate
-----END CERTIFICATE-----";
The PEM encoded private key of device
public const string CLIENT_KEY_PEM_A = @"-----BEGIN EC PRIVATE KEY-----
-----END EC PRIVATE KEY-----";
For a non-trivial system there should be a number of intermediate certificates. I have tried creating intermediate certificates for a device type, geography, application, customer and combinations of these. The first couple of times got it wrong so start with a field trial so that it isn’t so painful to go back and fix. (beware the sunk cost fallacy)
I found creating an intermediate certificate that could sign device certificates required a conf file for the basicConstraints and keyUsage configuration.
critical-The extension must be understood and processed by any application validating the certificate. If the application does not understand it, the certificate must be rejected.
CA:TRUE-This certificate is allowed to act as a Certificate Authority (CA), meaning it can sign other certificates.
pathlen:0-This CA can only issue end-entity (leaf) certificates and cannot issue further intermediate CA certificates.
keyCertSig- The certificate can be used to sign other certificates (i.e., it’s a CA certificate).
For production systems putting some thought into the Common name(CN), Organizational unit name(OU), Organization name(O), locality name(L), state or province name(S) and Country name(C)
Establishing a connection to the Azure Event Grid MQTT broker often failed which surprised me. Initially I didn’t have any retry logic which meant I wasted quite a bit of time trying to debug failed connections
The first message sent shortly after I powered up the device had the latitude and longitude of Null Island
The Asset Tracker UserApplicationId is 65002 and the payload is similar to the Swarm Eval Kit. I created some message payloads (location of Christchurch Cathedral) for testing.
namespace PayloadFormatter // Additional namespace for shortening interface when usage in formatter code
{
using System.Collections.Generic;
using Newtonsoft.Json.Linq;
public interface IFormatterUplink
{
public JObject Evaluate(IDictionary<string, string> properties, uint organisationId, uint deviceId, byte deviceType, ushort userApplicationId, JObject payloadJson, string payloadText, byte[] payloadBytes);
}
public interface IFormatterDownlink
{
public byte[] Evaluate(IDictionary<string, string> properties, uint organisationId, uint deviceId, byte deviceType, ushort userApplicationId, JObject payloadJson, string payloadText, byte[] payloadBytes);
}
}
The definitions of the uplink & downlink payload formatter evaluator interfaces have been updated and shifted to a new project.
Visual Studio 2022 Solution with payloadformatter maintenance application
I built a console application to help with developing and debugging uplink or downlink formatters. The application has a number of command line parameters which specify the formatter to be used, UserApplicationId, OrganizationId, DeviceType etc.
public class CommandLineOptions
{
[Option('d', "Direction", Required = true, HelpText = "Test Uplink or DownLink formatter")]
public string Direction { get; set; }
[Option('p', "filename", HelpText = "Uplink or Downlink Payload file name")]
public string PayloadFilename { get; set; } = string.Empty;
[Option('o', "OrganisationId", Required = true, HelpText = "Organisation unique identifier")]
public uint OrganizationId { get; set; }
[Option('i', "DeviceId", Required = true, HelpText = "Device unique identitifer")]
public uint DeviceId { get; set; }
[Option('t', "DeviceType", Required = true, HelpText = "Device type number")]
public byte DeviceType { get; set; }
[Option('u', "UserApplicationId", Required = true, HelpText = "User Application Id")]
public ushort UserApplicationId { get; set; }
[Option('h', "SwarmHiveReceivedAtUtc", HelpText = "Swarm Hive received at time UTC")]
public DateTime? SwarmHiveReceivedAtUtc { get; set; }
[Option('w', "UplinkWebHookReceivedAtUtc", HelpText = "Webhook received at time UTC")]
public DateTime? UplinkWebHookReceivedAtUtc { get; set; }
[Option('s', "Status", HelpText = "Uplink local file system file name")]
public byte? Status { get; set; }
[Option('c', "Client", HelpText = "Uplink local file system file name")]
public string Client { get; set; }
}
The downlink formatter (similar approach for uplink) loads the sample file as an array of bytes, then tries to convert it to text, and finally to JSON. Then the formatter code is “compiled” and the executed with the file payload and command line parameters.
private static async Task DownlinkFormatterCore(CommandLineOptions options)
{
Dictionary<string, string> properties = new Dictionary<string, string>();
string formatterFolder = Path.Combine(Environment.CurrentDirectory, "downlink");
Console.WriteLine($"Downlink- uplinkFormatterFolder: {formatterFolder}");
string formatterFile = Path.Combine(formatterFolder, $"{options.UserApplicationId}.cs");
Console.WriteLine($"Downlink- UserApplicationId: {options.UserApplicationId}");
Console.WriteLine($"Downlink- Payload formatter file: {formatterFile}");
PayloadFormatter.IFormatterDownlink evalulator;
try
{
evalulator = CSScript.Evaluator.LoadFile<PayloadFormatter.IFormatterDownlink>(formatterFile);
}
catch (CSScriptLib.CompilerException cex)
{
Console.Write($"Loading or compiling file:{formatterFile} failed Exception:{cex}");
return;
}
string payloadFilename = Path.Combine(formatterFolder, options.PayloadFilename);
Console.WriteLine($"Downlink- payloadFilename:{payloadFilename}");
byte[] uplinkBytes;
try
{
uplinkBytes = File.ReadAllBytes(payloadFilename);
}
catch (DirectoryNotFoundException dex)
{
Console.WriteLine($"Uplink payload filename directory {formatterFolder} not found:{dex}");
return;
}
catch (FileNotFoundException fnfex)
{
Console.WriteLine($"Uplink payload filename {payloadFilename} not found:{fnfex}");
return;
}
catch (FormatException fex)
{
Console.WriteLine($"Uplink payload file invalid format {payloadFilename} not found:{fex}");
return;
}
// See if payload can be converted to a string
string uplinkText = string.Empty;
try
{
uplinkText = Encoding.UTF8.GetString(uplinkBytes);
}
catch (FormatException fex)
{
Console.WriteLine("Encoding.UTF8.GetString failed:{0}", fex.Message);
}
// See if payload can be converted to JSON
JObject uplinkJson;
try
{
uplinkJson = JObject.Parse(uplinkText);
}
catch (JsonReaderException jrex)
{
Console.WriteLine("JObject.Parse failed Exception:{1}", jrex);
uplinkJson = new JObject();
}
Console.WriteLine("Properties");
foreach (var property in properties)
{
Console.WriteLine($"{property.Key}:{property.Value}");
}
// Transform the byte and optional text and JSON payload
Byte[] payload;
try
{
payload = evalulator.Evaluate(properties, options.OrganizationId, options.DeviceId, options.DeviceType, options.UserApplicationId, uplinkJson, uplinkText, uplinkBytes);
}
catch (Exception ex)
{
Console.WriteLine($"evalulatorUplink.Evaluate failed Exception:{ex}");
return;
}
Console.WriteLine("Payload");
Console.WriteLine(Convert.ToBase64String(payload));
}
The sample JSON payload is what would be sent by Azure IoT Central to a device to configure the fan speed
Azure IoT Central M138 Breakout device template with the Fan Status command selected
{
"FanStatus": 2
}
If the downlink payload formatter is compiled and executes successfully the Base64 representation output is displayed
using System;
using System.Collections.Generic;
using Newtonsoft.Json.Linq;
public class FormatterDownlink : PayloadFormatter.IFormatterDownlink
{
public byte[] Evaluate(IDictionary<string, string> properties, uint organisationId, uint deviceId, byte deviceType, ushort userApplicationId, JObject payloadJson, string payloadText, byte[] payloadBytes)
{
byte? status = payloadJson.Value<byte?>("FanStatus");
if ( status.HasValue )
{
return new byte[] { status.Value };
}
return new byte[]{};
}
}
If the downlink payload formatter syntax is incorrect e.g. { status.Value ; }; an error message with the line and column is displayed.
using System;
using System.Collections.Generic;
using Newtonsoft.Json.Linq;
public class FormatterDownlink : PayloadFormatter.IFormatterDownlink
{
public byte[] Evaluate(IDictionary<string, string> properties, uint organisationId, uint deviceId, byte deviceType, ushort userApplicationId, JObject payloadJson, string payloadText, byte[] payloadBytes)
{
byte? status = payloadJson.Value<byte?>("FanStatus");
if ( status.HasValue )
{
return new byte[] { status.Value ; };
}
return new byte[]{};
}
}
If the downlink payload formatter syntax is correct but execution fails (in the example code division by zero) an error message is displayed.
using System;
using System.Collections.Generic;
using Newtonsoft.Json.Linq;
public class FormatterDownlink : PayloadFormatter.IFormatterDownlink
{
public byte[] Evaluate(IDictionary<string, string> properties, uint organisationId, uint deviceId, byte deviceType, ushort userApplicationId, JObject payloadJson, string payloadText, byte[] payloadBytes)
{
byte? status = payloadJson.Value<byte?>("FanStatus");
if ( status.HasValue )
{
int divideByZero = 10;
divideByZero = divideByZero / 0;
return new byte[] { status.Value };
}
return new byte[]{};
}
}
The application now has a StartUpService which loads the Azure DeviceClient cache (Lazy Cache) in the background as the application starts up. If an uplink message is received from a SwarmDevice before, it has been loaded by the FunctionsStartup the DeviceClient information is cached and another connection to the Azure IoT Hub is not established.
I’m looking at building a webby user interface where users an interactivity list, create, edit, delete formatters with syntax highlighter support, and the executing the formatter with sample payloads.
Swarm Space Azure IoT Connector Identity Translation Gateway Architecture
This approach uses most of the existing building blocks, and that’s it no more changes.
https://json2csharp.com/
// Root myDeserializedClass = JsonConvert.DeserializeObject<Root>(myJsonResponse);
public class Root
{
public int packetId { get; set; }
public int deviceType { get; set; }
public int deviceId { get; set; }
public int userApplicationId { get; set; }
public int organizationId { get; set; }
public string data { get; set; }
public int len { get; set; }
public int status { get; set; }
public DateTime hiveRxTime { get; set; }
}
*/
public class UplinkPayload
{
[JsonProperty("packetId")]
public int PacketId { get; set; }
[JsonProperty("deviceType")]
public int DeviceType { get; set; }
[JsonProperty("deviceId")]
public int DeviceId { get; set; }
[JsonProperty("userApplicationId")]
public int UserApplicationId { get; set; }
[JsonProperty("organizationId")]
public int OrganizationId { get; set; }
[JsonProperty("data")]
[JsonRequired]
public string Data { get; set; }
[JsonProperty("len")]
public int Len { get; set; }
[JsonProperty("status")]
public int Status { get; set; }
[JsonProperty("hiveRxTime")]
public DateTime HiveRxTime { get; set; }
}
This class is used to “automagically” deserialise Delivery Webhook payloads. There is also some additional payload validation which discards test messages (not certain this is a good idea) etc.
//---------------------------------------------------------------------------------
// Copyright (c) December 2022, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.SwarmSpace.AzureIoT.Connector.Controllers
{
using System.Globalization;
using System.Text;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.Devices.Client;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
[ApiController]
[Route("api/[controller]")]
public class UplinkController : ControllerBase
{
private readonly ILogger<UplinkController> _logger;
private readonly IAzureIoTDeviceClientCache _azureIoTDeviceClientCache;
public UplinkController(ILogger<UplinkController> logger, IAzureIoTDeviceClientCache azureIoTDeviceClientCache)
{
_logger = logger;
_azureIoTDeviceClientCache = azureIoTDeviceClientCache;
}
[HttpPost]
public async Task<IActionResult> Uplink([FromBody] Models.UplinkPayload payload)
{
DeviceClient deviceClient;
_logger.LogDebug("Payload {0}", JsonConvert.SerializeObject(payload, Formatting.Indented));
if (payload.PacketId == 0)
{
_logger.LogWarning("Uplink-payload simulated DeviceId:{DeviceId}", payload.DeviceId);
return this.Ok();
}
if ((payload.UserApplicationId < Constants.UserApplicationIdMinimum) || (payload.UserApplicationId > Constants.UserApplicationIdMaximum))
{
_logger.LogWarning("Uplink-payload invalid User Application Id:{UserApplicationId}", payload.UserApplicationId);
return this.BadRequest($"Invalid User Application Id {payload.UserApplicationId}");
}
if ((payload.Len < Constants.PayloadLengthMinimum) || string.IsNullOrEmpty(payload.Data))
{
_logger.LogWarning("Uplink-payload.Data is empty PacketId:{PacketId}", payload.PacketId);
return this.Ok("payload.Data is empty");
}
Models.AzureIoTDeviceClientContext context = new Models.AzureIoTDeviceClientContext()
{
OrganisationId = payload.OrganizationId,
UserApplicationId = payload.UserApplicationId,
DeviceType = payload.DeviceType,
DeviceId = payload.DeviceId,
};
deviceClient = await _azureIoTDeviceClientCache.GetOrAddAsync(payload.DeviceId.ToString(), context);
JObject telemetryEvent = new JObject
{
{ "packetId", payload.PacketId},
{ "deviceType" , payload.DeviceType},
{ "DeviceID", payload.DeviceId },
{ "organizationId", payload.OrganizationId },
{ "ApplicationId", payload.UserApplicationId},
{ "ReceivedAtUtc", payload.HiveRxTime.ToString("s", CultureInfo.InvariantCulture) },
{ "DataLength", payload.Len },
{ "Data", payload.Data },
{ "Status", payload.Status },
};
// Send the message to Azure IoT Hub
using (Message ioTHubmessage = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryEvent))))
{
// Ensure the displayed time is the acquired time rather than the uploaded time.
ioTHubmessage.Properties.Add("iothub-creation-time-utc", payload.HiveRxTime.ToString("s", CultureInfo.InvariantCulture));
ioTHubmessage.Properties.Add("OrganizationId", payload.OrganizationId.ToString());
ioTHubmessage.Properties.Add("ApplicationId", payload.UserApplicationId.ToString());
ioTHubmessage.Properties.Add("DeviceId", payload.DeviceId.ToString());
ioTHubmessage.Properties.Add("deviceType", payload.DeviceType.ToString());
await deviceClient.SendEventAsync(ioTHubmessage);
_logger.LogInformation("Uplink-DeviceID:{deviceId} SendEventAsync success", payload.DeviceId);
}
return this.Ok();
}
}
}
The webhook was configured to “acknowledge messages on successful delivery”. I then checked my Delivery Method configuration with a couple of “Test” messages.
My Swarm Space Eval Kit arrived md-week and after some issues with jumper settings it started reporting position and status information.
The time-lapse functionality which continually uploaded images wasn’t that useful. I have another standalone application which has that functionality.
If an object with a label in the “PredictionLabelsOfInterest” and a score greater than PredicitionScoreThreshold was detected it was useful to have the option to upload the camera and/or marked-up (including objects below the threshold) image(s).