Swarm Space – Uplink Payload Message Creation Time

The Swarm Space satellite constellation doesn’t have continuous coverage (Jan 2023) so messages sent when there is no coverage are queued (default 48hrs) by the Swarm M138 Modem for transmission when a satellite passes overhead.

Satellite Passes with gap in coverage from 16:18 to 18:42 highlighted

In the Swarm Hive Delivery Method messages from the Swarm Eval Kit and Swarm Tracker in my backyard arriving in “clusters”.

Swarm Hive Delivery Methods webhook calls.

The messages in each “cluster” were processed by a payload formatter then forwarded to Azure IoT Central for processing. All the messages in a cluster had similar event creation times which was “breaking” graphs and device tracking maps. After running the application locally using Telerik Fiddler to try different payloads I realised that the Microsoft.Azure.Azure.Devices.Client message iothub-creation-time-utc property was set to the when the message was received by Swarm Space infrastructure.

_logger.LogDebug("Uplink-DeviceId:{0} PacketId:{1} TelemetryEvent before:{0}", payload.DeviceId, payload.PacketId, JsonConvert.SerializeObject(telemetryEvent, Formatting.Indented));

telemetryEvent = swarmSpaceFormatterUplink.Evaluate(telemetryEvent, payload.Data, payloadBytes, payloadText, payloadJson);

_logger.LogDebug("Uplink-DeviceId:{0} PacketId:{1} TelemetryEvent after:{0}", payload.DeviceId, payload.PacketId, JsonConvert.SerializeObject(telemetryEvent, Formatting.Indented));

// Send the message to Azure IoT Hub
using (Message ioTHubmessage = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryEvent))))
{
   // Ensure the displayed time is the acquired time rather than the uploaded time. 
   ioTHubmessage.Properties.Add("iothub-creation-time-utc", payload.HiveRxTime.ToString("s", CultureInfo.InvariantCulture));
   ioTHubmessage.Properties.Add("PacketId", payload.PacketId.ToString());
   ioTHubmessage.Properties.Add("OrganizationId", payload.OrganizationId.ToString());
   ioTHubmessage.Properties.Add("ApplicationId", payload.UserApplicationId.ToString());
   ioTHubmessage.Properties.Add("DeviceId", payload.DeviceId.ToString());
   ioTHubmessage.Properties.Add("deviceType", payload.DeviceType.ToString());

   await deviceClient.SendEventAsync(ioTHubmessage);

   _logger.LogInformation("Uplink-DeviceID:{deviceId} SendEventAsync success", payload.DeviceId);
}

The Swarm Eval Kit uplink (JSON) message generated by the sample firmware “d” field is the number of seconds since the Unix Epoch that the message payload was constructed.

Swarm Hive Messages with “d” field in the JSON payload highlighted
Online Unix Epoch Convertor displaying Unix Epoch 1672561286 in NZDT and UTC time

The revised 65355.cs payload formatter adds an “iothub-creation-time-utc” field to the TelemetryEvent

using System;
using System.Globalization;

using Newtonsoft.Json.Linq;

public class FormatterUplink : PayloadFormatter.IFormatterUplink
{
    public JObject Evaluate(JObject telemetryEvent, string payloadBase64, byte[] payloadBytes, string payloadText, JObject payloadJson)
    {
        if ((payloadText != "" ) && ( payloadJson != null))
        {
            JObject location = new JObject();

            location.Add("lat", payloadJson.GetValue("lt"));
            location.Add("lon", payloadJson.GetValue("ln"));
            location.Add("alt", payloadJson.GetValue("a"));

            telemetryEvent.Add("DeviceLocation", location);
        };

        telemetryEvent.Add("iothub-creation-time-utc", DateTimeOffset.FromUnixTimeSeconds((long)payloadJson.GetValue("d")).ToString("s", CultureInfo.InvariantCulture));

        return telemetryEvent;
    }
}

Which, if present is used to populate theMicrosoft.Azure.Azure.Devices.Client message iothub-creation-time-utc property

_logger.LogDebug("Uplink-DeviceId:{0} PacketId:{1} TelemetryEvent before:{0}", payload.DeviceId, payload.PacketId, JsonConvert.SerializeObject(telemetryEvent, Formatting.Indented));

telemetryEvent = swarmSpaceFormatterUplink.Evaluate(telemetryEvent, payload.Data, payloadBytes, payloadText, payloadJson);

.LogDebug("Uplink-DeviceId:{0} PacketId:{1} TelemetryEvent after:{0}", payload.DeviceId, payload.PacketId, JsonConvert.SerializeObject(telemetryEvent, Formatting.Indented));

Send the message to Azure IoT Hub
using (Message ioTHubmessage = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryEvent))))
{
   // Ensure the displayed time is the acquired time rather than the uploaded time. 
   ioTHubmessage.Properties.Add("PacketId", payload.PacketId.ToString());
   ioTHubmessage.Properties.Add("OrganizationId", payload.OrganizationId.ToString());
   ioTHubmessage.Properties.Add("UserApplicationId", payload.UserApplicationId.ToString());
   ioTHubmessage.Properties.Add("DeviceId", payload.DeviceId.ToString());
   ioTHubmessage.Properties.Add("deviceType", payload.DeviceType.ToString());

   if (telemetryEvent.ContainsKey("iothub-creation-time-utc"))
   {
      ioTHubmessage.Properties.Add("iothub-creation-time-utc",telemetryEvent.Value<string>("iothub-creation-time-utc"));
   }

   await deviceClient.SendEventAsync(ioTHubmessage);

   _logger.LogInformation("Uplink-DeviceID:{deviceId} SendEventAsync success", payload.DeviceId);
}

The Azure IoT Central message now had the correct timestamp and “event creation time” values.

AzureIoT Central “Raw Data” with valid timestamp and event creation times

I don’t think this is a good solution

The design of the payload formatters will have to be revisited

One thought on “Swarm Space – Uplink Payload Message Creation Time

  1. Pingback: Swarm Space – Uplink Payload Formatters revisited | devMobile's blog

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.