TTI V3 Gateway Device Provisioning Service(DPS) Concurrent Requests

While debugging The Things Industries(TTI) V3 connector on my desktop I had noticed that using an Azure IoT Hub device connection string was quite a bit faster than using the Azure Device Provisioning Service(DPS). The Azure Webjob connector was executing the requests sequentially which made the duration of the DPS call even more apparent.

To reduce the impact of the RegisterAsync call duration this Proof of Concept(PoC) code uses the System.Tasks.Threading library to execute each request in its own thread and then wait for all the requests to finish.

try
{
   int devicePage = 1;
   V3EndDevices endDevices = await endDeviceRegistryClient.ListAsync(
      applicationSetting.Key,
      field_mask_paths: Constants.DevicefieldMaskPaths,
      page: devicePage,
      limit: _programSettings.TheThingsIndustries.DevicePageSize,
      cancellationToken: stoppingToken);

   while ((endDevices != null) && (endDevices.End_devices != null)) // If no devices returns null rather than empty list
   {
      List<Task<bool>> tasks = new List<Task<bool>>();

      _logger.LogInformation("Config-ApplicationID:{0} start", applicationSetting.Key);

      foreach (V3EndDevice device in endDevices.End_devices)
      {
         if (DeviceAzureEnabled(device))
         {
            _logger.LogInformation("Config-ApplicationID:{0} DeviceID:{1} Device EUI:{2}", device.Ids.Application_ids.Application_id, device.Ids.Device_id, BitConverter.ToString(device.Ids.Dev_eui));

            tasks.Add(DeviceRegistration(device.Ids.Application_ids.Application_id,
                                       device.Ids.Device_id,
                                       _programSettings.ResolveDeviceModelId(device.Ids.Application_ids.Application_id, device.Attributes),
                                       stoppingToken));
         }
      }

      _logger.LogInformation("Config-ApplicationID:{0} Page:{1} processing start", applicationSetting.Key, devicePage);

      Task.WaitAll(tasks.ToArray(),stoppingToken);

      _logger.LogInformation("Config-ApplicationID:{0} Page:{1} processing finish", applicationSetting.Key, devicePage);

      endDevices = await endDeviceRegistryClient.ListAsync(
         applicationSetting.Key,
         field_mask_paths: Constants.DevicefieldMaskPaths,
         page: devicePage += 1,
         limit: _programSettings.TheThingsIndustries.DevicePageSize,
         cancellationToken: stoppingToken);
   }
   _logger.LogInformation("Config-ApplicationID:{0} finish", applicationSetting.Key);
}
catch (ApiException ex)
{
   _logger.LogError("Config-Application configuration API error:{0}", ex.StatusCode);
}

The connector application paginates the retrieval of device configuration from TTI API and a Task is created for each device returned in a page. In the Application Insights Trace logging the duration of a single page of device registrations was approximately the duration of the longest call.

There will be a tradeoff between device page size (resource utilisation by many threads) and startup duration (to many sequential page operations) which will need to be explored.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.