Random wanderings through Microsoft Azure esp. PaaS plumbing, the IoT bits, AI on Micro controllers, AI on Edge Devices, .NET nanoFramework, .NET Core on *nix and ML.NET+ONNX
Device Twin Explorer displaying telemetry from one of the Seeeduino devices
My integration uses only queued messages as often they won’t be delivered to the sensor node immediately, especially if the sensor node only sends an uplink message every 30 minutes/hour/day.
The confirmed flag should be used with care as the Azure IoT Hub messages may expire before a delivery Ack/Nack/Failed is received from the TTI.
PowerBI graph of temperature and humidity in my garage over 24 hours
Device explorer displaying a raw payload message which has been confirmed delivered
TTI device live data tab displaying raw payload in downlink message information tabAzure IoT Connector console application sending raw payload to sensor node with confirmation ack
Arduino monitor displaying received raw payload from TTI
If the Azure IoT Hub message payload is valid JSON it is copied into the payload decoded downlink message property. and if it is not valid JSON it assumed to be a Base64 encoded value and copied into the payload raw downlink message property.
try
{
// Split over multiple lines in an attempt to improve readability. A valid JSON string should start/end with {/} for an object or [/] for an array
if (!(payloadText.StartsWith("{") && payloadText.EndsWith("}"))
&&
(!(payloadText.StartsWith("[") && payloadText.EndsWith("]"))))
{
throw new JsonReaderException();
}
downlink.PayloadDecoded = JToken.Parse(payloadText);
}
catch (JsonReaderException)
{
downlink.PayloadRaw = payloadText;
}
Like the Azure IoT Central JSON validation I had to add a check that the string started with a “{” and finished with a “}” (a JSON object) or started with a “[” and finished with a “]” (a JSON array) as part of the validation process.
Device explorer displaying a JSON payload message which has been confirmed delivered
I normally wouldn’t use exceptions for flow control but I can’t see a better way of doing this.
TTI device live data tab displaying JSON payload in downlink message information tabAzure IoT Connector console application sending JSON payload to sensor node with confirmation ackArduino monitor displaying received JSON payload from TTI
The build in TTI decoder only supports downlink decoded payloads with property names “value_0” through “value_x” custom encoders may support other property names.
The first step was to display the temperature and barometric pressure values from the Seeedstudio Grove BMP180 attached to my sensor node.
Sensor node displaying temperature and barometric pressure values
Azure IoT Central temperature and barometric pressure telemetry configuration
Azure IoT Central Telemetry Dashboard displaying temperature and barometric pressure values
The next step was to configure a simple Azure IoT Central command to send to the sensor node. This was a queued request with no payload. An example of this sort of command would be a request for a sensor node to reboot or turn on an actuator.
My integration uses only offline queued commands as often messages won’t be delivered to the sensor node immediately, especially if the sensor node only sends a message every half hour/hour/day. The confirmed flag should be used with care as the Azure IoT Hub messages may expire before a delivery Ack/Nack/Failed is received from the TTI and it consumes downlink bandwidth.
if (message.Properties.ContainsKey("method-name"))
{
}
Azure IoT Central command with out a request payload value command configuration
To send a downlink message, TTI needs a LoRaWAN port number (plus optional queue, confirmed and priority values) which can’t be provided via the Azure IoT Central command setup so these values are configured in the app.settings file.
Each TTI application has zero or more Azure IoT Central command configurations which supply the port, confirmed, priority and queue settings.
Azure IoT Central simple command dashboardAzure IoT Central simple command initiationAzure IoT TTI connector application sending a simple command to my sensor nodeSensor node display simple command information. The note message payload is empty
The next step was to configure a more complex Azure IoT Central command to send to the sensor node. This was a queued request with a single value payload. An example of this sort of command could be setting the speed of a fan or the maximum temperature of a freezer for an out of band (OOB) notification to be sent.
Azure IoT Central single value command configuration
The value_0 settings are for the minimum temperature the value_1 settings are for the maximum temperature value.
Azure IoT Central single value command initiationAzure IoT TTI connector application sending a single value command to my sensor nodeSensor node displaying single value command information. There are two downlink messages and each payload contains a single value
The single value command payload contains the textual representation of the value e.g. “true”/”false” or “1.23” which are also valid JSON. This initially caused issues as I was trying to splice a single value into the decoded payload.
I had to add a check that the string started with a “{” and finished with a “}” (a JSON object) or started with a “[” and finished with a “]” (a JSON array) as part of the validation process.
For a single value command the payload decoded has a single property with the method-name value as the name and the payload as the value. For a command with a JSON payload the message payload is copied into the PayloadDecoded.
I normally wouldn’t use exceptions for flow control but I can’t see a better way of doing this.
try
{
// Split over multiple lines to improve readability
if (!(payloadText.StartsWith("{") && payloadText.EndsWith("}"))
&&
(!(payloadText.StartsWith("[") && payloadText.EndsWith("]"))))
{
throw new JsonReaderException();
}
downlink.PayloadDecoded = JToken.Parse(payloadText);
}
catch (JsonReaderException)
{
try
{
JToken value = JToken.Parse(payloadText);
downlink.PayloadDecoded = new JObject(new JProperty(methodName, value));
}
catch (JsonReaderException)
{
downlink.PayloadDecoded = new JObject(new JProperty(methodName, payloadText));
}
}
The final step was to configure an another Azure IoT Central command with a JSON payload to send to the sensor node. A “real-world” example of this sort of command would be setting the minimum and maximum temperatures of a freezer in a single downlink message.
Azure IoT Central JSON payload command setup
Azure IoT Central JSON payload command payload configuration
Azure IoT TTI connector application sending a JSON payload command to my sensor node
Sensor node displaying JSON command information. There is a single payload which contains a two values
The build in TTI decoder only supports downlink decoded payloads with property names “value_0” through “value_x” which results in some odd command names and JSON payload property names. (Custom encoders may support other property names). Case sensitivity of some configuration values also tripped me up.
return DeviceClient.CreateFromConnectionString(connectionString, deviceId,
new ITransportSettings[]
{
new AmqpTransportSettings(TransportType.Amqp_Tcp_Only)
{
PrefetchCount = 0,
AmqpConnectionPoolSettings = new AmqpConnectionPoolSettings()
{
Pooling = true,
}
}
});
I hadn’t noticed this issue in my Azure IoT The Things Network Integration because I hadn’t built support for C2D messaging. After some trial and error I figured out the issue was the PrefetchCount initialisation.
return DeviceClient.CreateFromConnectionString(connectionString, deviceId,
new ITransportSettings[]
{
new AmqpTransportSettings(TransportType.Amqp_Tcp_Only)
{
AmqpConnectionPoolSettings = new AmqpConnectionPoolSettings()
{
Pooling = true,
}
}
});
Even though the Service Bus APIs do not directly expose such an option today, a lower-level AMQP protocol client can use the link-credit model to turn the “pull-style” interaction of issuing one unit of credit for each receive request into a “push-style” model by issuing a large number of link credits and then receive messages as they become available without any further interaction. Push is supported through the MessagingFactory.PrefetchCount or MessageReceiver.PrefetchCount property settings. When they are non-zero, the AMQP client uses it as the link credit.
n this context, it’s important to understand that the clock for the expiration of the lock on the message inside the entity starts when the message is taken from the entity, not when the message is put on the wire. Whenever the client indicates readiness to receive messages by issuing link credit, it is therefore expected to be actively pulling messages across the network and be ready to handle them. Otherwise the message lock may have expired before the message is even delivered. The use of link-credit flow control should directly reflect the immediate readiness to deal with available messages dispatched to the receiver.
In the Azure IoT Hub SDK the prefetch count is set to 50 (around line 57) and throws an exception if less that zero (around line 90) and there is some information about tuning the prefetch value for Azure Service Bus.
“You are correct, the pre-fetch count is used to set the link credit over AMQP. What this signifies is the max. no. of messages that can be “in-flight” from the service to the client, at any given time. (This value defaults to 50 for the IoT Hub .NET client). The client specifies its link-credit, that the service must respect. In simplest terms, any time the service sends a message to the client, it decrements the link credit, and will continue sending messages until linkCredit > 0. Once the client acknowledges the message, it will increment the link credit.”
In summary if Prefetch count is set to zero on startup in my application no messages will be sent to the client….
After trialing a couple of different approaches I have removed the AzureSettingsDefault. If an application has a connectionstring configured that is used, if there is not one then the DPS configuration is used, if there are neither currently the application logs an error. In the future I will look at adding a configuration option to make the application optionally shutdown
After configuring, deploying and then operating my The Things Network(TTN) V2 gateway I have made some changes to my The Things Industries(TTI) V3 gateway.
Using Azure KeyVault to store configuration was an interesting learning exercise but made configuration difficult for users, so for the initial V3 version(s) I have dropped support and reverted to an app.settings file.
The V2 gateway used an Azure HTTP Trigger function to process TTN uplink messages which were placed into an Azure Storage Queue for processing by an Azure Queue Trigger function. This was complex to deploy and caused message ordering problems when multiple instances of the storage queue trigger function where spun up to process a backlog of messages.
The V2 Gateway only provisioned devices with the Azure Device Provisioning Service on the first uplink message. This made it difficult to process Downlink messages as there was no Azure DeviceClient connection for devices which hadn’t sent a message. The V3 gateway uses the TTN API to enumerate the devices in each TTN Application configured in the app.settings.json file. For each application a Message Queue Telemetry Transport(MQTT) (using MQTTNet) connection is opened for receiving uplink messages, sending downlink messages and tracking the progress of downlink messages. Then for each TTN Device a connection is establish to the specified Azure IoT Hub to enable Cloud to Device(C2D) and Device to Cloud messaging.
With so many components the V2 gateway was difficult to debug, so the V3 version runs locally as a console application and in Azure as an Azure continuous Webjob
The amount of diagnostic logging sent to Azure Application Insights was making it difficult to identify and then diagnose issues so the way logging is implemented has been revisited.
TTI V3 Gateway running as a console application on my desktop
Azure IoT integration can be configured at the Device (TTN Device “azureintegration” attribute).
TTN Device AzureIntegration Attribute
Then falls back to the Application default (TTN application “azureintegrationdevicedefault” attribute).
Then falls back to the “DeviceIntegrationDefault” setting for the Application then finally “DeviceIntegrationDefault” setting for the webjob the in the app.settings.json file
My Azure IoT Central client application displays the generated message including the decoded payload field which is used by the built in Low Power Protocol(LPP) decoder/encoder and other custom encoders/decoders.
Azure IoT Central commands for TTN/TTI integration
From the “Device Commands” form I can send commands and a queued commands which have float parameters or object parameters which contain one or more float values in a JSON payload.
For commands which call the methodHander which was been registered by calling SetMethodDefaultHandlerAsync the request payload can be JSON or plain text. If the payload is valid JSON it is “grafted”(couldn’t think of a better word) into the decoded_payload field. If the payload is not valid a JSON object with the method name as the “name” and the text payload as the value is added the decoded_payload.
private static async Task<MethodResponse> MethodCallbackDefaultHandler(MethodRequest methodRequest, object userContext)
{
AzureIoTMethodHandlerContext receiveMessageHandlerConext = (AzureIoTMethodHandlerContext)userContext;
Console.WriteLine($"Default handler method {methodRequest.Name} was called.");
Console.WriteLine($"Payload:{methodRequest.DataAsJson}");
Console.WriteLine();
if (string.IsNullOrWhiteSpace(methodRequest.Name))
{
Console.WriteLine($" Method Request Name null or white space");
return new MethodResponse(400);
}
string payloadText = Encoding.UTF8.GetString(methodRequest.Data);
if (string.IsNullOrWhiteSpace(payloadText))
{
Console.WriteLine($" Payload null or white space");
return new MethodResponse(400);
}
// At this point would check to see if Azure DeviceClient is in cache, this is so nasty
if ( String.Compare( methodRequest.Name, "Analog_Output_1", true) ==0 )
{
Console.WriteLine($" Device not found");
return new MethodResponse(UTF8Encoding.UTF8.GetBytes("Device not found"), 404);
}
JObject payload;
if (IsValidJSON(payloadText))
{
payload = JObject.Parse(payloadText);
}
else
{
payload = new JObject
{
{ methodRequest.Name, payloadText }
};
}
string downlinktopic = $"v3/{receiveMessageHandlerConext.ApplicationId}@{receiveMessageHandlerConext.TenantId}/devices/{receiveMessageHandlerConext.DeviceId}/down/push";
DownlinkPayload downlinkPayload = new DownlinkPayload()
{
Downlinks = new List<Downlink>()
{
new Downlink()
{
Confirmed = false,
//PayloadRaw = messageBody,
PayloadDecoded = payload,
Priority = DownlinkPriority.Normal,
Port = 10,
/*
CorrelationIds = new List<string>()
{
methodRequest.LockToken
}
*/
}
}
};
Console.WriteLine($"TTN Topic :{downlinktopic}");
Console.WriteLine($"TTN downlink JSON :{JsonConvert.SerializeObject(downlinkPayload, Formatting.Indented)}");
return new MethodResponse(200);
}
Configuration of unqueued Commands with a typed payloadThe output of my test harness for a Command for a typed payloadConfiguring fields of object payload(JSON)
A JSON request payload also supports downlink messages with more that one value.
The output of my test harness for a Command with an object payload(JSON)
For queued commands which call the ReceiveMessageHandler which has was registered by calling SetReceiveMessageHandler the request payload is JSON or plain text.
When I initiated an Analog queued command the message handler was invoked with the name of the command capability (Analog_Output_2) in a message property called “method-name”. For a typed parameter the message content was a string representation of the value. For an object parameter the payload contains a JSON representation of the request field(s)
The output of my test harness for a Queued Command with a typed payload
A JSON request payload supports downlink message with more that one value.
The output of my test harness for a Queued Command with an object payload(JSON)
The context information for both comments and queued commands provides additional information required to construct the MQTT topic for publishing the downlink messages.
If the device is not known the Abandon method will be called immediately. For command messages Completed will be called as soon as the message is “sent”
With object based parameters the request JSON could contain more than one value though the validation of user provided information didn’t appear to be as robust.
Object parameter schema definition
I “migrated” my third preconfigured device to the CommandRequest template to see how the commands with Request parameters interacted with my PoC application.
After “migrating” my device I went back and created a Template view so I could visualise the simulated telemetry from my PoC application and provide a way to initiate commands (Didn’t really need four command tiles as they all open the Device commands form).
CommandRequest device template default view
From the Device Commands form I could send commands and a queued commands which had analog or digital parameters.
Device Three Command Tab
When I initiated an Analog non-queued command the default method handler was invoked with the name of the command capability (Analog_Output_1) as the method name and the payload contained a JSON representation of the request values(s). With a typed parameter a string representation of the value was in the message payload. With a typed parameter a string representation of the value was in the message payload rather than JSON.
Console application displaying Analog request and Analog Request queued commands
When I initiated an Analog queued command the message handler was invoked with the name of the command capability (Analog_Output_2) in a message property called “method-name” and the payload contained a JSON representation of the request value(s). With a typed parameter a string representation of the value was in the message payload rather than JSON.
When I initiated a Digital non-queued command the default method handler was invoked with the name of the command capability (Digital_Output_1) as the method name and the payload contained a JSON representation of the request values(s). With a typed parameter a string representation of the value was in the message payload rather than JSON.
Console application displaying Digital request and Digital Request queued commands
When I initiated a Digital queued command the message handler was invoked with the name of the command capability(Digital_Output_2) in a message property called “method-name” and the payload contained a JSON representation of the request value(s). With a typed parameter a string representation of the value was in the message payload rather than JSON.
The validation of user input wasn’t as robust as I expected, with problems selecting checkboxes with a mouse when there were several Boolean fields. I often had to click on a nearby input field and use the TAB button to navigate to the desired checkbox. I also had problems with ISO 8601 format date validation as the built in Date Picker returned a month, day, year date which was not editable and wouldn’t pass validation.
The next logical step would be to look at commands with a Response parameter but as the MQTT interface is The Things Network(TTN) and The Things Industries(TTI) is asynchronous and devices reporting every 5 minutes to a couple of times a day there could be a significant delay between sending a message and receiving an optional delivery confirmation or response.
I have been struggling with making The Things Network(TTN) and The Things Industries(TTI) uplink/downlink messages work well Azure IoT Central. To explore different messaging approaches I have built a proof of Concept(PoC) application which simulates TTN/TTI connectivity to an Azure IoT Hub, or Azure IoT Central.
This blog post is about queued and non queued Cloud to Device(C2D) commands without request or response parameters. I have mostly used non queued commands in other projects (my Azure IoT HubLoRa and RF24L01 gateways) to “Restart” devices etc..
From the Device Commands tab I can could non queued and a queued commands
Device Two Commands tab
When I sent a non-queued command the default method handler was invoked with the name of the command capability (Digital_Output_0) as the method name and an empty payload. In the Azure IoT Central interface I couldn’t see any difference for successful (HTTP 200 OK) or failure (HTTP 400 Bad Request or HTTP 404 Not Found) responses. If the application was not running the command failed immediately.
When I sent a queued command the message handler was invoked with the name of the command capability(Digital_Output_1) in a message property called “method-name” and the payload contained only an “@” character.
Console application displaying queued call
If the application was not running the command was queued until the Console application was started. When the console application was running and AbandonAsync was called rather than CompleteAsync the message was retried 10 times. If RejectAsync was called rather than CompleteAsync the message was deleted from the queue and not retried. There didn’t appear to be any difference for the displayed Azure IoT Central or Azure IoT Hub explorer results when AbandonAsync or RejectAsync were called.
I also created a personal dashboard to visualise the telemetry data and initiate commands. The way the two commands were presented on the dashboard was quite limited so I will go back to the documentation and see what I missed
I have been struggling with making The Things Network(TTN) and The Things Industries(TTI) uplink/downlink messages Azure IoT Central compatible. To explore the messaging approaches used I have built a proof of Concept(PoC) application which simulates TTN/TTI connectivity to an Azure IoT Hub, or Azure IoT Central.
I then “migrated” the first device to my BasicTelemetry template
Migrating a device to TelemetryBasic template
I then went back and created a Template view to visualise the telemetry from my console application.
TelemetryBasic device template default view
Then I configured a preview device so the template view was populated with “realistic” data.
TelemetryBasic device template default view configuring a device as data source
The console application simulates a digital input (random true/false), analog input (random value between 0.0 and 1.0) and a Global Positioning System(GPS) location (Christchurch Anglican Cathedral with a random latitude, longitude and altitude offset) .
The first step was to add the The Things Network(TTN)V3 Tennant ID to the context information as it is required for the downlink Message Queue Telemetry Transport (MQTT) publish topic.
namespace devMobile.TheThingsNetwork.Models
{
public class AzureIoTHubReceiveMessageHandlerContext
{
public string TenantId { get; set; }
public string DeviceId { get; set; }
public string ApplicationId { get; set; }
}
}
To send a message to a LoRaWAN device in addition to the payload, TTN needs the port number and optionally a confirmation required flag, message priority, queueing type and correlation ids.
With my implementation the confirmation required flag, message priority, and queueing type are Azure IoT Hub message properties and the messageid is used as a correlation id.
private async static Task AzureIoTHubClientReceiveMessageHandler(Message message, object userContext)
{
bool confirmed;
byte port;
DownlinkPriority priority;
string downlinktopic;
try
{
AzureIoTHubReceiveMessageHandlerContext receiveMessageHandlerConext = (AzureIoTHubReceiveMessageHandlerContext)userContext;
DeviceClient deviceClient = (DeviceClient)DeviceClients.Get(receiveMessageHandlerConext.DeviceId);
if (deviceClient == null)
{
Console.WriteLine($" UplinkMessageReceived unknown DeviceID: {receiveMessageHandlerConext.DeviceId}");
await deviceClient.RejectAsync(message);
return;
}
using (message)
{
Console.WriteLine();
Console.WriteLine();
Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Azure IoT Hub downlink message");
Console.WriteLine($" ApplicationID: {receiveMessageHandlerConext.ApplicationId}");
Console.WriteLine($" DeviceID: {receiveMessageHandlerConext.DeviceId}");
#if DIAGNOSTICS_AZURE_IOT_HUB
Console.WriteLine($" Cached: {DeviceClients.Contains(receiveMessageHandlerConext.DeviceId)}");
Console.WriteLine($" MessageID: {message.MessageId}");
Console.WriteLine($" DeliveryCount: {message.DeliveryCount}");
Console.WriteLine($" EnqueuedTimeUtc: {message.EnqueuedTimeUtc}");
Console.WriteLine($" SequenceNumber: {message.SequenceNumber}");
Console.WriteLine($" To: {message.To}");
#endif
string messageBody = Encoding.UTF8.GetString(message.GetBytes());
Console.WriteLine($" Body: {messageBody}");
#if DOWNLINK_MESSAGE_PROPERTIES_DISPLAY
foreach (var property in message.Properties)
{
Console.WriteLine($" Key:{property.Key} Value:{property.Value}");
}
#endif
if (!message.Properties.ContainsKey("Confirmed"))
{
Console.WriteLine(" UplinkMessageReceived missing confirmed property");
await deviceClient.RejectAsync(message);
return;
}
if (!bool.TryParse(message.Properties["Confirmed"], out confirmed))
{
Console.WriteLine(" UplinkMessageReceived confirmed property invalid");
await deviceClient.RejectAsync(message);
return;
}
if (!message.Properties.ContainsKey("Priority"))
{
Console.WriteLine(" UplinkMessageReceived missing priority property");
await deviceClient.RejectAsync(message);
return;
}
if (!Enum.TryParse(message.Properties["Priority"], true, out priority))
{
Console.WriteLine(" UplinkMessageReceived priority property invalid");
await deviceClient.RejectAsync(message);
return;
}
if (priority == DownlinkPriority.Undefined)
{
Console.WriteLine(" UplinkMessageReceived priority property undefined value invalid");
await deviceClient.RejectAsync(message);
return;
}
if (!message.Properties.ContainsKey("Port"))
{
Console.WriteLine(" UplinkMessageReceived missing port number property");
await deviceClient.RejectAsync(message);
return;
}
if (!byte.TryParse( message.Properties["Port"], out port))
{
Console.WriteLine(" UplinkMessageReceived port number property invalid");
await deviceClient.RejectAsync(message);
return;
}
if ((port < Constants.PortNumberMinimum) || port > (Constants.PortNumberMaximum))
{
Console.WriteLine($" UplinkMessageReceived port number property invalid value must be between {Constants.PortNumberMinimum} and {Constants.PortNumberMaximum}");
await deviceClient.RejectAsync(message);
return;
}
if (!message.Properties.ContainsKey("Queue"))
{
Console.WriteLine(" UplinkMessageReceived missing queue property");
await deviceClient.RejectAsync(message);
return;
}
switch(message.Properties["Queue"].ToLower())
{
case "push":
downlinktopic = $"v3/{receiveMessageHandlerConext.ApplicationId}@{receiveMessageHandlerConext.TenantId}/devices/{receiveMessageHandlerConext.DeviceId}/down/push";
break;
case "replace":
downlinktopic = $"v3/{receiveMessageHandlerConext.ApplicationId}@{receiveMessageHandlerConext.TenantId}/devices/{receiveMessageHandlerConext.DeviceId}/down/replace";
break;
default:
Console.WriteLine(" UplinkMessageReceived missing queue property invalid value");
await deviceClient.RejectAsync(message);
return;
}
DownlinkPayload Payload = new DownlinkPayload()
{
Downlinks = new List<Downlink>()
{
new Downlink()
{
Confirmed = confirmed,
PayloadRaw = messageBody,
Priority = priority,
Port = port,
CorrelationIds = new List<string>()
{
message.MessageId
}
}
}
};
var mqttMessage = new MqttApplicationMessageBuilder()
.WithTopic(downlinktopic)
.WithPayload(JsonConvert.SerializeObject(Payload))
.WithAtLeastOnceQoS()
.Build();
await mqttClient.PublishAsync(mqttMessage);
// Need to look at confirmation requirement ack, nack maybe failed & sent
await deviceClient.CompleteAsync(message);
Console.WriteLine();
}
}
catch (Exception ex)
{
Debug.WriteLine("UplinkMessageReceived failed: {0}", ex.Message);
}
}
To “smoke test”” my implementation I used Azure IoT Explorer to send a C2D telemetry message
Azure IoT Hub Explorer send message form with payload and message properties
The PoC console application then forwarded the message to TTN using MQTT to be sent(which fails)
PoC application sending message then displaying result
The TTN live data display shows the message couldn’t be delivered because my test LoRaWAN device has not been activiated.
TTN Live Data display with message delivery failure