The Things Network HTTP Azure IoT Central Integration

This post is an overview of the Azure IoT Central configuration required to process The Things Network(TTN) HTTP integration uplink messages. I have assumed that the reader is already reasonably familiar with these products. There is an overview of configuring TTN HTTP integration in my “Simplicating and securing the HTTP handler” post.

The first step is to copy the IDScope from the Device connection blade.

Device connection blade

Then copy one of the primary or secondary keys

For more complex deployment the ApplicationEnrollmentGroupMapping configuration enables The Things Network(TTN) devices to be provisioned using different GroupEnrollment keys based on the applicationid in the Uplink message which initiates their provisoning.

"DeviceProvisioningService": {
      "GlobalDeviceEndpoint": "global.azure-devices-provisioning.net",
      "IDScope": "",
      "EnrollmentGroupSymmetricKeyDefault": "TopSecretKey",
      "DeviceProvisioningPollingDelay": 500,
      "ApplicationEnrollmentGroupMapping": {
         "Application1": "TopSecretKey1",
         "Application2": "TopSecretKey2"
      }
   }

Shortly after the first uplink message from a TTN device is processed, it will listed in the “Unassociated devices” blade with the DevEUI as the Device ID.

Unassociated devices blade

The device can then be associated with an Azure IoT Central Device Template.

Unassociated devices blade showing recently associated device

The device template provides for the mapping of uplink message payload_fields to measurements. In this example the payload field has been generated by the TTN HTTP integration Cayenne Low Power Protocol(LPP) decoder. Many LoRaWAN devices use LPP to minimise the size of the network payload.

Azure IoT Central Device template blade

Once the device has been associated with a template a user friendly device name etc. can be configured.

Azure IoT Central Device properties blade

In the telemetry event payload sent to Azure IoT Central there are some extra fields to help with debugging and tracing.

// Assemble the JSON payload to send to Azure IoT Hub/Central.
log.LogInformation($"{messagePrefix} Payload assembly start");
JObject telemetryEvent = new JObject();
try
{
   JObject payloadFields = (JObject)payloadObect.payload_fields;
   telemetryEvent.Add("HardwareSerial", payloadObect.hardware_serial);
   telemetryEvent.Add("Retry", payloadObect.is_retry);
   telemetryEvent.Add("Counter", payloadObect.counter);
   telemetryEvent.Add("DeviceID", payloadObect.dev_id);
   telemetryEvent.Add("ApplicationID", payloadObect.app_id);
   telemetryEvent.Add("Port", payloadObect.port);
   telemetryEvent.Add("PayloadRaw", payloadObect.payload_raw);
   telemetryEvent.Add("ReceivedAtUTC", payloadObect.metadata.time);

   // If the payload has been unpacked in TTN backend add fields to telemetry event payload
   if (payloadFields != null)
   {
      foreach (JProperty child in payloadFields.Children())
      {
         EnumerateChildren(telemetryEvent, child);
      }
   }
}
catch (Exception ex)
{
   log.LogError(ex, $"{messagePrefix} Payload processing or Telemetry event assembly failed");
   throw;
}

Azure IoT Central has mapping functionality which can be used to display the location of a device.

Azure Device

The format of the location payload generated by the TTN LPP decoder is different to the one required by Azure IoT Central. I have added temporary code (“a cost effective modification to expedite deployment” aka. a hack) to format the TelemetryEvent payload so it can be processed.

if (token.First is JValue)
{
   // Temporary dirty hack for Azure IoT Central compatibility
   if (token.Parent is JObject possibleGpsProperty)
   {
      if (possibleGpsProperty.Path.StartsWith("GPS", StringComparison.OrdinalIgnoreCase))
      {
         if (string.Compare(property.Name, "Latitude", true) == 0)
         {
            jobject.Add("lat", property.Value);
         }
         if (string.Compare(property.Name, "Longitude", true) == 0)
         {
            jobject.Add("lon", property.Value);
         }
         if (string.Compare(property.Name, "Altitude", true) == 0)
         {
            jobject.Add("alt", property.Value);
         }
      }
   }
   jobject.Add(property.Name, property.Value);
}

I need review the IoT Plug and Play specification documentation to see what other payload transformations maybe required.

I did observe that if a device had not reported its position the default location was zero degrees latitude and zero degrees longitude which is about 610 KM south of Ghana and 1080 KM west of Gabon in the Atlantic Ocean.

Azure IoT Central mapping default position

After configuring a device template, associating my devices with the template, and modifying each device’s properties I could create a dashboard to view the temperature and humidity information returned by my Seeeduino LoRaWAN devices.

Azure IoT Central dashboard

The Things Network HTTP Integration Part6

Provisioning Devices on demand.

For development and testing being able to provision an individual device is really useful, though for Azure IoT Central it is not easy (especially with the deprecation of DPS-KeyGen). With an Azure IoT Hub device connection strings are available in the portal which is convenient but not terribly scalable.

Azure IoT Hub is integrated with, and Azure IoT Central forces the use of the Device Provisioning Service(DPS) which is designed to support the management of 1000’s of devices.

My HTTP Integration for The Things Network(TTN) is intended to support many devices and integrate with Azure IoT Central so I built yet another “nasty” console application to explore how the DPS works. The DPS also supports device attestation with a Trusted Platform Module(TPM) but this approach was not suitable for my application.

My command-line application supports individual and group enrollments with Symmetric Key Attestation and it can also generate group enrollment device keys.

class Program
{
   private const string GlobalDeviceEndpoint = "global.azure-devices-provisioning.net";

   static async Task Main(string[] args)
   {
      string registrationId;
...   
      registrationId = args[1];

      switch (args[0])
      {
         case "e":
         case "E":
            string scopeId = args[2];
            string symmetricKey = args[3];

            Console.WriteLine($"Enrolllment RegistrationID:{ registrationId} ScopeID:{scopeId}");
            await Enrollement(registrationId, scopeId, symmetricKey);
            break;
         case "k":
         case "K":
            string primaryKey = args[2];
            string secondaryKey = args[3];

            Console.WriteLine($"Enrollment Keys RegistrationID:{ registrationId}");
            GroupEnrollementKeys(registrationId, primaryKey, secondaryKey);
            break;
         default:
            Console.WriteLine("Unknown option");
            break;
      }
      Console.WriteLine("Press <enter> to exit");
      Console.ReadLine();
   }

   static async Task Enrollement(string registrationId, string scopeId, string symetricKey)
   {
      try
      {
         using (var securityProvider = new SecurityProviderSymmetricKey(registrationId, symetricKey, null))
         {
            using (var transport = new ProvisioningTransportHandlerAmqp(TransportFallbackType.TcpOnly))
            {
               ProvisioningDeviceClient provClient = ProvisioningDeviceClient.Create(GlobalDeviceEndpoint, scopeId, securityProvider, transport);

               DeviceRegistrationResult result = await provClient.RegisterAsync();

               Console.WriteLine($"Hub:{result.AssignedHub} DeviceID:{result.DeviceId} RegistrationID:{result.RegistrationId} Status:{result.Status}");
               if (result.Status != ProvisioningRegistrationStatusType.Assigned)
               {
                  Console.WriteLine($"DeviceID{ result.Status} already assigned");
               }

               IAuthenticationMethod authentication = new DeviceAuthenticationWithRegistrySymmetricKey(result.DeviceId, (securityProvider as SecurityProviderSymmetricKey).GetPrimaryKey());

               using (DeviceClient iotClient = DeviceClient.Create(result.AssignedHub, authentication, TransportType.Amqp))
               {
                  Console.WriteLine("DeviceClient OpenAsync.");
                  await iotClient.OpenAsync().ConfigureAwait(false);
                  Console.WriteLine("DeviceClient SendEventAsync.");
                  await iotClient.SendEventAsync(new Message(Encoding.UTF8.GetBytes("TestMessage"))).ConfigureAwait(false);
                  Console.WriteLine("DeviceClient CloseAsync.");
                  await iotClient.CloseAsync().ConfigureAwait(false);
               }
            }
         }
      }
      catch (Exception ex)
      {
         Console.WriteLine(ex.Message);
      }
   }

   static void GroupEnrollementKeys(string registrationId, string primaryKey, string secondaryKey)
   {
      string primaryDeviceKey = ComputeDerivedSymmetricKey(Convert.FromBase64String(primaryKey), registrationId);
      string secondaryDeviceKey = ComputeDerivedSymmetricKey(Convert.FromBase64String(secondaryKey), registrationId);

      Console.WriteLine($"RegistrationID:{registrationId}");
      Console.WriteLine($" PrimaryDeviceKey:{primaryDeviceKey}");
      Console.WriteLine($" SecondaryDeviceKey:{secondaryDeviceKey}");
   }

   public static string ComputeDerivedSymmetricKey(byte[] masterKey, string registrationId)
   {
      using (var hmac = new HMACSHA256(masterKey))
      {
         return Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(registrationId)));
      }
   }
}

I have five seeeduino LoRaWAN and a single Seeeduino LoRaWAN W/GPS device leftover from another project so I created a SeeeduinoLoRaWAN enrollment group.

DPS Enrollment Group configuration

Initially the enrollment group had no registration records so I ran my command-line application to generate group enrollment keys for one of my devices.

Device registration before running my command line application

Then I ran the command-line application with my scopeID, registrationID (LoRaWAN deviceEUI) and the device group enrollment key I had generated in the previous step.

Registering a device and sending a message to the my Azure IoT Hub

After running the command line application the device was visible in the enrollment group registration records.

Device registration after running my command line application

Provisioning a device with an individual enrollment has a different workflow. I had to run my command-line application with the RegistrationID, ScopeID, and one of the symmetric keys from the DPS individual enrollment device configuration.

DPS Individual enrollment configuration

A major downside to an individual enrollment is either the primary or the secondary symmetric key for the device has to be deployed on the device which could be problematic if the device has no secure storage.

With a group enrollment only the registration ID and the derived symmetric key have to be deployed on the device which is more secure.

Registering a device and sending a message to the my Azure IoT Hub

In Azure IoT Explorer I could see messages from both my group and individually enrolled devices arriving at my Azure IoT hub

After some initial issues I found DPS was quite reliable and surprisingly easy to configure. I did find the DPS ProvisioningDeviceClient.RegisterAsync method sometimes took several seconds to execute which may have some ramifications when my application is doing this on demand.