Swarm Space – Uplink Payload Message Creation Time

The Swarm Space satellite constellation doesn’t have continuous coverage (Jan 2023) so messages sent when there is no coverage are queued (default 48hrs) by the Swarm M138 Modem for transmission when a satellite passes overhead.

Satellite Passes with gap in coverage from 16:18 to 18:42 highlighted

In the Swarm Hive Delivery Method messages from the Swarm Eval Kit and Swarm Tracker in my backyard arriving in “clusters”.

Swarm Hive Delivery Methods webhook calls.

The messages in each “cluster” were processed by a payload formatter then forwarded to Azure IoT Central for processing. All the messages in a cluster had similar event creation times which was “breaking” graphs and device tracking maps. After running the application locally using Telerik Fiddler to try different payloads I realised that the Microsoft.Azure.Azure.Devices.Client message iothub-creation-time-utc property was set to the when the message was received by Swarm Space infrastructure.

_logger.LogDebug("Uplink-DeviceId:{0} PacketId:{1} TelemetryEvent before:{0}", payload.DeviceId, payload.PacketId, JsonConvert.SerializeObject(telemetryEvent, Formatting.Indented));

telemetryEvent = swarmSpaceFormatterUplink.Evaluate(telemetryEvent, payload.Data, payloadBytes, payloadText, payloadJson);

_logger.LogDebug("Uplink-DeviceId:{0} PacketId:{1} TelemetryEvent after:{0}", payload.DeviceId, payload.PacketId, JsonConvert.SerializeObject(telemetryEvent, Formatting.Indented));

// Send the message to Azure IoT Hub
using (Message ioTHubmessage = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryEvent))))
{
   // Ensure the displayed time is the acquired time rather than the uploaded time. 
   ioTHubmessage.Properties.Add("iothub-creation-time-utc", payload.HiveRxTime.ToString("s", CultureInfo.InvariantCulture));
   ioTHubmessage.Properties.Add("PacketId", payload.PacketId.ToString());
   ioTHubmessage.Properties.Add("OrganizationId", payload.OrganizationId.ToString());
   ioTHubmessage.Properties.Add("ApplicationId", payload.UserApplicationId.ToString());
   ioTHubmessage.Properties.Add("DeviceId", payload.DeviceId.ToString());
   ioTHubmessage.Properties.Add("deviceType", payload.DeviceType.ToString());

   await deviceClient.SendEventAsync(ioTHubmessage);

   _logger.LogInformation("Uplink-DeviceID:{deviceId} SendEventAsync success", payload.DeviceId);
}

The Swarm Eval Kit uplink (JSON) message generated by the sample firmware “d” field is the number of seconds since the Unix Epoch that the message payload was constructed.

Swarm Hive Messages with “d” field in the JSON payload highlighted
Online Unix Epoch Convertor displaying Unix Epoch 1672561286 in NZDT and UTC time

The revised 65355.cs payload formatter adds an “iothub-creation-time-utc” field to the TelemetryEvent

using System;
using System.Globalization;

using Newtonsoft.Json.Linq;

public class FormatterUplink : PayloadFormatter.IFormatterUplink
{
    public JObject Evaluate(JObject telemetryEvent, string payloadBase64, byte[] payloadBytes, string payloadText, JObject payloadJson)
    {
        if ((payloadText != "" ) && ( payloadJson != null))
        {
            JObject location = new JObject();

            location.Add("lat", payloadJson.GetValue("lt"));
            location.Add("lon", payloadJson.GetValue("ln"));
            location.Add("alt", payloadJson.GetValue("a"));

            telemetryEvent.Add("DeviceLocation", location);
        };

        telemetryEvent.Add("iothub-creation-time-utc", DateTimeOffset.FromUnixTimeSeconds((long)payloadJson.GetValue("d")).ToString("s", CultureInfo.InvariantCulture));

        return telemetryEvent;
    }
}

Which, if present is used to populate theMicrosoft.Azure.Azure.Devices.Client message iothub-creation-time-utc property

_logger.LogDebug("Uplink-DeviceId:{0} PacketId:{1} TelemetryEvent before:{0}", payload.DeviceId, payload.PacketId, JsonConvert.SerializeObject(telemetryEvent, Formatting.Indented));

telemetryEvent = swarmSpaceFormatterUplink.Evaluate(telemetryEvent, payload.Data, payloadBytes, payloadText, payloadJson);

.LogDebug("Uplink-DeviceId:{0} PacketId:{1} TelemetryEvent after:{0}", payload.DeviceId, payload.PacketId, JsonConvert.SerializeObject(telemetryEvent, Formatting.Indented));

Send the message to Azure IoT Hub
using (Message ioTHubmessage = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryEvent))))
{
   // Ensure the displayed time is the acquired time rather than the uploaded time. 
   ioTHubmessage.Properties.Add("PacketId", payload.PacketId.ToString());
   ioTHubmessage.Properties.Add("OrganizationId", payload.OrganizationId.ToString());
   ioTHubmessage.Properties.Add("UserApplicationId", payload.UserApplicationId.ToString());
   ioTHubmessage.Properties.Add("DeviceId", payload.DeviceId.ToString());
   ioTHubmessage.Properties.Add("deviceType", payload.DeviceType.ToString());

   if (telemetryEvent.ContainsKey("iothub-creation-time-utc"))
   {
      ioTHubmessage.Properties.Add("iothub-creation-time-utc",telemetryEvent.Value<string>("iothub-creation-time-utc"));
   }

   await deviceClient.SendEventAsync(ioTHubmessage);

   _logger.LogInformation("Uplink-DeviceID:{deviceId} SendEventAsync success", payload.DeviceId);
}

The Azure IoT Central message now had the correct timestamp and “event creation time” values.

AzureIoT Central “Raw Data” with valid timestamp and event creation times

I don’t think this is a good solution

The design of the payload formatters will have to be revisited

Swarm Space – Uplink Payload formatter caching and files

The payload formatters of my Azure IoT Hub Cloud Identity Translation Gateway use CS-Script and even a simple one was taking more than half a second to compile each time it was called.

using System;
using System.Globalization;

using Newtonsoft.Json.Linq;

public class FormatterUplink : PayloadFormatter.IFormatterUplink
{
    public JObject Evaluate(JObject telemetryEvent, string payloadBase64, byte[] payloadBytes, string payloadText, JObject payloadJson)
    {
        if ((payloadText != "" ) && ( payloadJson != null))
        {
            JObject location = new JObject();

            location.Add("lat", payloadJson.GetValue("lt"));
            location.Add("lon", payloadJson.GetValue("ln"));
            location.Add("alt", payloadJson.GetValue("a"));

            telemetryEvent.Add("Location", location);
        };

        return telemetryEvent;
    }
}

The Swarm Eval Kit default message has a userApplicationId of 65335

{"ln":123.456,"si":0.0,"bi":0.2,"sv":0.152,"lt":-12.345,"bv":4.032,"d":1671704370,"n":2,"a":9.0,"s":1.0,"c":208.0,"r":-94,"ti":0.032}

The 65355.cs payload formatter adds an Azure IoT Central compatible location to the telemetry payload.

Azure IoT Central uplink telemetry message payload

The formatter files are currently part of the SwarmSpaceAzureIoTConnector project (moving to Azure Blob Storage) so are configured as “content” (bonus syntax highlighting works) and “copy if newer” so they are included in the deployment package.

Visual Studio 2022 Sample payload formatter

I used Alastair Crabtrees’s LazyCache to store compiled payload formatters with Uplink/Downlink + UserApplicationId as the cache key.

public class FormatterCache : IFormatterCache
{
    private readonly ILogger<FormatterCache> _logger;
    private readonly Models.ApplicationSettings _applicationSettings;
    private readonly static IAppCache _payloadFormatters = new CachingService();

    public FormatterCache(ILogger<FormatterCache>logger, IOptions<Models.ApplicationSettings> applicationSettings)
    {
        _logger = logger;
        _applicationSettings = applicationSettings.Value;
    }

    public async Task<IFormatterUplink> UplinkGetAsync(int userApplicationId)
    {
        IFormatterUplink payloadFormatterUplink = await _payloadFormatters.GetOrAddAsync<PayloadFormatter.IFormatterUplink>($"U{userApplicationId}", (ICacheEntry x) => UplinkLoadAsync(userApplicationId), memoryCacheEntryOptions);

        return payloadFormatterUplink;
    }

    private async Task<IFormatterUplink> UplinkLoadAsync(int userApplicationId)
    {
        string payloadformatterFilePath = $"{_applicationSettings.PayloadFormattersUplinkFilePath}\\{userApplicationId}.cs";

        if (!File.Exists(payloadformatterFilePath))
        {
            _logger.LogInformation("PayloadFormatterUplink- UserApplicationId:{0} PayloadFormatterPath:{1} not found using default:{2}", userApplicationId, payloadformatterFilePath, _applicationSettings.PayloadFormatterUplinkDefault);

            return CSScript.Evaluator.LoadFile<PayloadFormatter.IFormatterUplink>(_applicationSettings.PayloadFormatterUplinkDefault);
        }

        _logger.LogInformation("PayloadFormatterUplink- UserApplicationId:{0} loading PayloadFormatterPath:{1}", userApplicationId, payloadformatterFilePath);

        return CSScript.Evaluator.LoadFile<PayloadFormatter.IFormatterUplink>(payloadformatterFilePath);
    }
...
}

The default uplink and downlink formatters are configured in application settings and are used when a UserApplicationId specific formatter is not configured.

Fiddler Composer illustrating compiled formatter timings before and after caching

Swarm Space – Uplink Payload formatter Proof of Concept(PoC)

My Azure IoT Hub Cloud Identity Translation Gateway will support the translation of Base64 encoded uplink payloads to Javascript Object Notation (JSON) so they can be processed by Azure IoT Hub client applications and Azure IoT Central. This PoC uses CS-Script by Oleg Shilo to transform the Swarm Eval Kit Base64 encoded JSON uplink messages.

Swarm Hive message list with a message payload

A sample decoded (JSON) Swarm Eval Kit uplink message

{"ln":123.456,"si":0.0,"bi":0.2,"sv":0.152,"lt":-12.345,"bv":4.032,"d":1671704370,"n":2,"a":9.0,"s":1.0,"c":208.0,"r":-94,"ti":0.032}

A Webhook Delivery method forwards uplink messages to my Azure IoT Hub Cloud Identity Translation Gateway.

Swarm Hive Delivery configuration with recent uplink messages

My first hard-coded payload formatter adds an Azure IoT Central compatible location to the telemetry event payload.

const string codeSwarmSpaceUplinkFormatterCode = @"
   using Newtonsoft.Json.Linq;

   public class UplinkFormatter : PayloadFormatter.ISwarmSpaceFormatterUplink
   {
       public JObject Evaluate(JObject telemetryEvent, string payloadBase64, byte[] payloadBytes, string payloadText, JObject payloadJson)
       {
           if ((payloadText != """" ) && ( payloadJson != null))
           {
               JObject location = new JObject() ;

               location.Add(""Lat"", payloadJson.GetValue(""lt""));
               location.Add(""Lon"", payloadJson.GetValue(""ln""));
               location.Add(""Alt"", payloadJson.GetValue(""a""));

               telemetryEvent.Add( ""location"", location);
           };

           return telemetryEvent;
       }
   }";
}

The PayloadFormatter namespace was added to reduce the length of the payload formatter C# interface declarations.

namespace PayloadFormatter 
{
    using Newtonsoft.Json.Linq;

    public interface ISwarmSpaceFormatterUplink
    {
        public JObject Evaluate(JObject telemetry, string payloadBase64, byte[] payloadBytes, string payloadText, JObject payloadJson);
    }

    public interface ISwarmSpaceFormatterDownlink
    {
        public string Evaluate(JObject payloadJson, string payloadText, byte[] payloadBytes, string payloadBase64);
    }
}

namespace devMobile.IoT.SwarmSpace.AzureIoT.Connector
{
    using System.Threading.Tasks;
    using Microsoft.Extensions.Logging;

    using CSScriptLib;

    using PayloadFormatter;

    public interface ISwarmSpaceFormatterCache
    {
        public Task<ISwarmSpaceFormatterUplink> PayloadFormatterGetOrAddAsync(int userApplicationId);

    }

    public class SwarmSpaceFormatterCache : ISwarmSpaceFormatterCache
    {
        private readonly ILogger<SwarmSpaceFormatterCache> _logger;

        public SwarmSpaceFormatterCache(ILogger<SwarmSpaceFormatterCache>logger)
        {
            _logger = logger;
        }

        public async Task<ISwarmSpaceFormatterUplink> PayloadFormatterGetOrAddAsync(int deviceId)
        {
            return CSScript.Evaluator.LoadCode<PayloadFormatter.ISwarmSpaceFormatterUplink>(codeSwarmSpaceUplinkFormatterCode);
        }
...
}

The parameters of the formatter are Base64 encoded, textual and a Newtonsoft JObject representations of the uplink payload and a telemetry event populated with some uplink message metadata.

Azure IoT Central uplink telemetry message payload

The initial “compile” of an uplink formatter was taking approximately 2.1 seconds so they will be “compiled” on demand and cached in a Dictionary with the UserApplicationId as the key. A default uplink formatter will be used when a UserApplicationId specific uplink formatter is not configured.

Swarm Space – Azure IoT FromDevice with webhooks

The initial versions of the Swarm Space Azure Cloud Identity Gateway were based on my The Things Industries(TTI) Azure IoT Connector which used six HTTP Triggered Azure Functions. My Swarm Space Azure IoT connector only has one webhook endpoint so a .NET Core WebAPI with controllers based solution appeared to be more practical. The first step was to get some sample JavaScript Object Notation(JSON) uplink message payloads with the SwarmSpace-From Device with Webhooks project.

{
  "packetId": 0,
  "deviceType": 1,
  "deviceId": 0,
  "userApplicationId": 0,
  "organizationId": 65760,
  "data": "VGhpcyBpcyBhIHRlc3QgbWVzc2FnZS4gVGhlIHBhY2tldElkIGFuZCBkZXZpY2VJZCBhcmUgbm90IHBvcHVsYXRlZCwgYnV0IHdpbGwgYmUgZm9yIGEgcmVhbCBtZXNzYWdlLg==",
  "len": 100,
  "status": 0,
  "hiveRxTime": "2022-11-29T04:52:06"
}

I used JSON2CSharp to generate an initial version of a Plain Old CLR(ComonLanguage Runtime) Object(POCO) to deserialise the Delivery Webhook payload.

 https://json2csharp.com/
    
    // Root myDeserializedClass = JsonConvert.DeserializeObject<Root>(myJsonResponse);
    public class Root
    {
        public int packetId { get; set; }
        public int deviceType { get; set; }
        public int deviceId { get; set; }
        public int userApplicationId { get; set; }
        public int organizationId { get; set; }
        public string data { get; set; }
        public int len { get; set; }
        public int status { get; set; }
        public DateTime hiveRxTime { get; set; }
    }
*/

I then “tweaked” the JSON2CSharp class

 public class UplinkPayload
    {
        [JsonProperty("packetId")]
        public int PacketId { get; set; }

        [JsonProperty("deviceType")]
        public int DeviceType { get; set; }

        [JsonProperty("deviceId")]
        public int DeviceId { get; set; }

        [JsonProperty("userApplicationId")]
        public int UserApplicationId { get; set; }

        [JsonProperty("organizationId")]
        public int OrganizationId { get; set; }

        [JsonProperty("data")]
        [JsonRequired]
        public string Data { get; set; }

        [JsonProperty("len")]
        public int Len { get; set; }

        [JsonProperty("status")]
        public int Status { get; set; }

        [JsonProperty("hiveRxTime")]
        public DateTime HiveRxTime { get; set; }
    }

This class is used to “automagically” deserialise Delivery Webhook payloads. There is also some additional payload validation which discards test messages (not certain this is a good idea) etc.

//---------------------------------------------------------------------------------
// Copyright (c) December 2022, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//---------------------------------------------------------------------------------
namespace devMobile.IoT.SwarmSpace.AzureIoT.Connector.Controllers
{
    using System.Globalization;
    using System.Text;
    using System.Threading.Tasks;

    using Microsoft.AspNetCore.Mvc;
    using Microsoft.Azure.Devices.Client;
    using Microsoft.Extensions.Logging;

    using Newtonsoft.Json;
    using Newtonsoft.Json.Linq;

    [ApiController]
    [Route("api/[controller]")]
    public class UplinkController : ControllerBase
    {
        private readonly ILogger<UplinkController> _logger;
        private readonly IAzureIoTDeviceClientCache _azureIoTDeviceClientCache;

        public UplinkController(ILogger<UplinkController> logger, IAzureIoTDeviceClientCache azureIoTDeviceClientCache)
        {
            _logger = logger;
            _azureIoTDeviceClientCache = azureIoTDeviceClientCache;
        }

        [HttpPost]
        public async Task<IActionResult> Uplink([FromBody] Models.UplinkPayload payload)
        {
            DeviceClient deviceClient;

            _logger.LogDebug("Payload {0}", JsonConvert.SerializeObject(payload, Formatting.Indented));

            if (payload.PacketId == 0)
            {
                _logger.LogWarning("Uplink-payload simulated DeviceId:{DeviceId}", payload.DeviceId);

                return this.Ok();
            }

            if ((payload.UserApplicationId < Constants.UserApplicationIdMinimum) || (payload.UserApplicationId > Constants.UserApplicationIdMaximum))
            {
                _logger.LogWarning("Uplink-payload invalid User Application Id:{UserApplicationId}", payload.UserApplicationId);

                return this.BadRequest($"Invalid User Application Id {payload.UserApplicationId}");
            }

            if ((payload.Len < Constants.PayloadLengthMinimum) || string.IsNullOrEmpty(payload.Data))
            {
                _logger.LogWarning("Uplink-payload.Data is empty PacketId:{PacketId}", payload.PacketId);

                return this.Ok("payload.Data is empty");
            }

            Models.AzureIoTDeviceClientContext context = new Models.AzureIoTDeviceClientContext()
            {
                OrganisationId = payload.OrganizationId,
                UserApplicationId = payload.UserApplicationId,
                DeviceType = payload.DeviceType,
                DeviceId = payload.DeviceId,
            };

            deviceClient = await _azureIoTDeviceClientCache.GetOrAddAsync(payload.DeviceId.ToString(), context);

            JObject telemetryEvent = new JObject
            {
                { "packetId", payload.PacketId},
                { "deviceType" , payload.DeviceType},
                { "DeviceID", payload.DeviceId },
                { "organizationId", payload.OrganizationId },
                { "ApplicationId", payload.UserApplicationId},
                { "ReceivedAtUtc", payload.HiveRxTime.ToString("s", CultureInfo.InvariantCulture) },
                { "DataLength", payload.Len },
                { "Data", payload.Data },
                { "Status", payload.Status },
            };

            // Send the message to Azure IoT Hub
            using (Message ioTHubmessage = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryEvent))))
            {
                // Ensure the displayed time is the acquired time rather than the uploaded time. 
                ioTHubmessage.Properties.Add("iothub-creation-time-utc", payload.HiveRxTime.ToString("s", CultureInfo.InvariantCulture));
                ioTHubmessage.Properties.Add("OrganizationId", payload.OrganizationId.ToString());
                ioTHubmessage.Properties.Add("ApplicationId", payload.UserApplicationId.ToString());
                ioTHubmessage.Properties.Add("DeviceId", payload.DeviceId.ToString());
                ioTHubmessage.Properties.Add("deviceType", payload.DeviceType.ToString());

                await deviceClient.SendEventAsync(ioTHubmessage);

                _logger.LogInformation("Uplink-DeviceID:{deviceId} SendEventAsync success", payload.DeviceId);
            }

            return this.Ok();
        }
    }
}

I initially debugged and tested the Uplink controller with Telerik Fiddler using sample payloads captured with the SwarmSpace-From Device with Webhooks project.

Using Telerik Fiddler to make test delivery webhook calls

Which I could then inspect with Azure IoT Explorer as they arrived

Azure IoT Explorer displaying a test message

The next step was to create a new Delivery Method

Swarm delivery webhook creation

Configured to call my Uplink controller endpoint.

Swarm delivery webhook configuration

The webhook was configured to “acknowledge messages on successful delivery”. I then checked my Delivery Method configuration with a couple of “Test” messages.

My Swarm Space Eval Kit arrived md-week and after some issues with jumper settings it started reporting position and status information.

Swarm Eval Kit in my backyard

The first position was just of the coast of West Africa(null island)

Swarm Map centered on Null Island

After the Global Positioning System(GPS) receiver got a good fix the location of the Eval Kit was in the middle of my backyard.

Azure IoT Explorer displaying payload with good latitude and longitude
Swarm Map displaying the location of my device (zoomed out)

Swarm Space – Payload formatters with CS-Script

My Azure IoT Hub Cloud Identity Translation Gateway needs to support the translation of Base64 encoded uplink payloads to Javascript Object Notation (JSON) and downlink payloads to Base64 encoded from Javascript Object Notation (JSON) . This so uplink and downlink messages can be processed and generated by Azure IoT Hub connected and Azure IoT Central applications.

To format uplink and downlink messages I had been looking at CS-Script by Oleg Shilo which is a Common Language Runtime(CLR) based scripting system that uses European Computer Manufacturers Association (ECMA)-compliant C# as a programming language.

I started with a modified version of the first sample on Github.

public class Samples
{
    const string codeMethod = @"
        int Multiply(int a, int b)
        {
            return a * b;
        }";

    public void Execute1()
    {
       dynamic script = CSScript.Evaluator.LoadMethod(codeMethod);

        int result = script.Multiply(3, 2);

        Console.WriteLine($"Product 1:{result}");
    }
...
internal class Program
{
    static void Main(string[] args)
    {
        new Samples().Execute1();
...
        Console.WriteLine($"Press Enter to exit");
        Console.ReadLine();
    }
}

I then modified it to use a C# interface and the application failed with an exception

CSScriptLib.CompilerException
  HResult=0x80131600
  Message=(2,39): error CS0246: The type or namespace name 'IMultiplier' could not be found (are you missing a using directive or an assembly reference?)

  Source=CSScriptLib
  StackTrace:
   at CSScriptLib.RoslynEvaluator.Compile(String scriptText, String scriptFile, CompileInfo info)
   at CSScriptLib.EvaluatorBase`1.LoadCode[T](String scriptText, Object[] args)
   at devMobile.IoT.SwarmSpace.AzureIoT.PayloadFormatterCSScript.Samples.Execute2A() in C:\Users\BrynLewis\source\repos\SwarmSpaceAzureIoT\PayloadFormatterCSScipt\Program.cs:line 90
   at devMobile.IoT.SwarmSpace.AzureIoT.PayloadFormatterCSScript.Program.Main(String[] args) in C:\Users\BrynLewis\source\repos\SwarmSpaceAzureIoT\PayloadFormatterCSScipt\Program.cs:line 375

After some trial and error, I figured out I had the namespace wrong

const string codeClassA = @"
    public class Calculator : devMobile.IoT.SwarmSpace.AzureIoT.PayloadFormatterCSScript.IMultiplier
    {
        public int Multiply(int a, int b)
        {

            return a * b;
        }
    }";

public void Execute2A()
{
    IMultiplier multiplierA = CSScript.Evaluator.LoadCode<IMultiplier>(codeClassA);

    Console.WriteLine($"Product 2A:{multiplierA.Multiply(3, 2)} - Press Enter to exit");
}

The long namespace would have been a pain in the arse (PITA) for users creating payload formatters and after some experimentation I added another interface with a short namespace. (Not certain this is a good idea).

namespace PayloadFormatter // Additional namespace for shortening interface for formatters
{
    public interface IMultiplier
    {
        int Multiply(int a, int b);
    }
}
...
public void Execute2B()
{
      PayloadFormatter.IMultiplier multiplierB = CSScript.Evaluator.LoadCode<PayloadFormatter.IMultiplier>(codeClassB);

     Console.WriteLine($"Product 2B:{multiplierB.Multiply(3, 2)} - Press Enter to exit");
}

I then wanted to figure out how to limit the namepaces the script has access to

const string codeClassDebug = @"
    using System.Diagnostics;

    public class Calculator : devMobile.IoT.SwarmSpace.AzureIoT.PayloadFormatterCSScript.IMultiplier
    {
        public int Multiply(int a, int b)
        {
           Debug.WriteLine(""Oops""); // Comment out the using System.Diagnostics;

            return a * b;
        }
    }";

public void Execute3()
{
    CSScript.Evaluator.Reset(true);

    IMultiplier multiplier = CSScript.Evaluator
        .LoadCode<IMultiplier>(codeClassDebug);

    int result = multiplier.Multiply(6, 2);

    Console.WriteLine($"Product 3:{result}");
}

The CSScript.Evaluator.Reset(true); removes all of the “default” references but a using directive could make namespaces available, so this needs some more investigation

The next step was to build the simplest possible payload formatter a “pipe” which displayed the text encoded in Base64 string.

const string codeSwarmSpaceFormatterPipe = @"
    public class SwarmSpaceFormatter:devMobile.IoT.SwarmSpace.AzureIoT.PayloadFormatterCSScript.ISwarmSpaceFormatterPipe
    {
        public string Pipe(string payloadBase64)
        {
            var payloadBase64Bytes = System.Convert.FromBase64String(payloadBase64);

             return System.Text.Encoding.UTF8.GetString(payloadBase64Bytes);
        }
    }";
...
public void Execute4()
{
    ISwarmSpaceFormatterPipe SwarmSpaceFormatter = CSScript.Evaluator
           ...
                        .LoadCode<ISwarmSpaceFormatterPipe>(codeSwarmSpaceFormatterPipe);

    string payload = SwarmSpaceFormatter.Pipe(PayloadBase64);

    Console.WriteLine($"Pipe:{payload}");
}

The Base64 encoded uplink payloads will have to be converted to JSON and the downlink JSON payloads will have to be converted to Base64 encoded binary, so I created an uplink and downlink formatters.

public void Execute5()
{
    string namespaces = $"using Newtonsoft.Json.Linq;using System;\n";
    string code = namespaces + codeSwarmSpaceFormatter;

    JObject telemetry = new JObject
    {
            { "ApplicationID", 12345 },
            { "DeviceID", 54321 },
            { "DeviceType", 2 },
            { "ReceivedAtUtc", DateTime.UtcNow.ToString("s", CultureInfo.InvariantCulture) },
    };

    ISwarmSpaceFormatter SwarmSpaceFormatter = CSScript.Evaluator.LoadCode<ISwarmSpaceFormatter>(code);

    string pipePayload = SwarmSpaceFormatter.Pipe(PayloadBase64);

    Console.WriteLine($"Pipe:{pipePayload}");
    Console.WriteLine("");


    JObject uplinkPayload = SwarmSpaceFormatter.Uplink(telemetry, PayloadBase64, Convert.FromBase64String(PayloadBase64));

    Console.WriteLine($"Uplink:{uplinkPayload}");
    Console.WriteLine("");

    JObject command = new JObject
    {
        {"Temperature", 1},
    };

    string downlinkPayload = SwarmSpaceFormatter.Downlink(command);

    Console.WriteLine($"Downlink:{downlinkPayload}");
    Console.WriteLine("");
}

I found that having both the byte array and Base64 encoded representation of the uplink payloads was useful. The first formatter converts the temperature field of the downlink payload into a four byte array then reverses the array to illustrate how packed byte payloads could be constructed.

const string codeSwarmSpaceFormatter1 = @"
    public class SwarmSpaceFormatter : devMobile.IoT.SwarmSpace.AzureIoT.PayloadFormatterCSScript.ISwarmSpaceFormatter
    {
        public string Pipe(string payloadBase64)
        {
            var payloadBase64Bytes = System.Convert.FromBase64String(payloadBase64);

            return System.Text.Encoding.UTF8.GetString(payloadBase64Bytes);
       }

        public JObject Uplink(JObject telemetryEvent, string payloadBase64, byte[] payloadBytes)
        {
            var payloadBase64Bytes = System.Convert.FromBase64String(payloadBase64);

            telemetryEvent.Add(""PayloadBase64"", payloadBase64Bytes);
            telemetryEvent.Add(""PayloadBytes"",System.Text.Encoding.UTF8.GetString(payloadBytes));

            return telemetryEvent;
        }

        public string Downlink(JObject command)
        {
            int temperature = command.Value<int>(""Temperature"");

            return System.Convert.ToBase64String(BitConverter.GetBytes(temperature));
        }
    }";

const string codeSwarmSpaceFormatter2 = @"
    public class SwarmSpaceFormatter:devMobile.IoT.SwarmSpace.AzureIoT.PayloadFormatterCSScript.ISwarmSpaceFormatter
    {
        public string Pipe(string payloadBase64)
        {
            var payloadBase64Bytes = System.Convert.FromBase64String(payloadBase64);

            return System.Text.Encoding.UTF8.GetString(payloadBase64Bytes);
        }

        public JObject Uplink(JObject telemetryEvent, string payloadBase64, byte[] payloadBytes)
        {
            var payloadBase64Bytes = System.Convert.FromBase64String(payloadBase64);

            telemetryEvent.Add(""PayloadBase64"", payloadBase64Bytes);
            telemetryEvent.Add(""PayloadBytes"",System.Text.Encoding.UTF8.GetString(payloadBytes));

            return telemetryEvent;
        }

        public string Downlink(JObject command)
        {
            int temperature = command.Value<int>(""Temperature"");

            byte[] temperatureBytes = BitConverter.GetBytes(temperature);

            Array.Reverse(temperatureBytes);

            return System.Convert.ToBase64String(temperatureBytes);
        }
    }";
...
public void Execute6()
{
    string namespaces = $"using Newtonsoft.Json.Linq;using System;\n";
    string code1 = namespaces + codeSwarmSpaceFormatter1;
    string code2 = namespaces + codeSwarmSpaceFormatter2;

    JObject telemetry = new JObject
    {
        { "ApplicationID", 12345 },
        { "DeviceID", 54321 },
        { "DeviceType", 2 },
        { "ReceivedAtUtc", DateTime.UtcNow.ToString("s", CultureInfo.InvariantCulture) },
    };

    var formatters = new Dictionary<string, ISwarmSpaceFormatter>();

    Console.WriteLine($"Evaluator start");
    DateTime evaluatorStartAtUtc = DateTime.UtcNow;

    ISwarmSpaceFormatter SwarmSpaceFormatter1 = CSScript.Evaluator
                                  .LoadCode<ISwarmSpaceFormatter>(code1);

    ISwarmSpaceFormatter SwarmSpaceFormatter2 = CSScript.Evaluator
                                  .LoadCode<ISwarmSpaceFormatter>(code2);

    Console.WriteLine($"Evaluator:{DateTime.UtcNow - evaluatorStartAtUtc}");
    Console.WriteLine("");

    Console.WriteLine($"Evaluation start");
    DateTime evaluationStartUtc = DateTime.UtcNow;

    formatters.Add("F1", SwarmSpaceFormatter1);
    formatters.Add("F2", SwarmSpaceFormatter2);

    JObject command = new JObject
    {
        {"Temperature", 1},
    }; 

    ISwarmSpaceFormatter downlinkPayload;
    downlinkPayload = formatters["F1"];
    Console.WriteLine($"Downlink F1:{downlinkPayload.Downlink(command)}");
  
    downlinkPayload = formatters["F2"];
    Console.WriteLine($"Downlink F2:{downlinkPayload.Downlink(command)}");
  
    Console.WriteLine($"Evaluation:{DateTime.UtcNow - evaluationStartUtc}");
    Console.WriteLine("");

    const int iterations = 100;
    Console.WriteLine($"Evaluations start {iterations}");
    DateTime evaluationsStartUtc = DateTime.UtcNow;

    for (int i = 1; i <= iterations; i++)
    {
        JObject command1 = new JObject
        {
            {"Temperature", 1},
        };

        downlinkPayload = formatters["F1"];
        Console.WriteLine($" Downlink F1:{downlinkPayload.Downlink(command1)}");
       
        downlinkPayload = formatters["F2"];
        Console.WriteLine($" Downlink F2:{downlinkPayload.Downlink(command1)}");
    }

    Console.WriteLine($"Evaluations:{iterations} Took:{DateTime.UtcNow - evaluationsStartUtc}");
}

On my development box the initial “compile” of each function was taking approximately 2.1 seconds so I cached the “compiled” formatters in a dictionary so they could be reused. Cached in the dictionary executing the two formatters 100 times took approximately 15 milliseconds (which is close to native .NET performance).

Compatibility

To check that the CS-Script tooling could run on a machine without the .NET 6 Software Development Kit (SDK) I tested the application on a laptop which had a “fresh” install of Windows 10.

CS-Script application failing due to missing .NET 6 runtime
Installing the .NET 6 Runtime
CS-Script application running after .NET runtime installation

The CS-Script library is pretty amazing and has made the development of uplink and downlink payload formatters significantly less complex than I was expecting.

Swarm Space – FromDevice with webhooks

I modified my TTI V3 Connector Azure Storage Queues project which uses Azure Functions HTTP Triggers to put messages into Azure Storage Queues to process Swarm FromDevice Webhook messages.

First step was to configure a webhook with the Swarm dashboard

Swarm dashboard webhooks configuration

I configured the webhook, and to “acknowledge messages on successful delivery”. Then checked my configuration with a couple of “Test” messages.

Swarm dashboard webhook configuration

The Swagger API documentation has methods for configuring endpoints which can be called by an application.

Swagger API Documentation for managing endpoints

I queued a couple of messages on my Satellite Transceiver Breakout and when the next satellite passed overhead, shortly after they were visible in the Swarm Dashboard Messages tab.

Swarm Dashboard with test and live fromdevice messages

The messages were also delivered to an Azure Storage Queue, and I could view them with Azure Storage Explorer.

Azure Storage Explorer displaying a webhook message payload

Swarm Space – Azure IoT Basic Client

To figure out how to poll the Swarm Hive API I have built yet another “nasty” Proof of Concept (PoC) which gets ToDevice and FromDevice messages. Initially I have focused on polling as the volume of messages from my single device is pretty low (WebHooks will be covered in a future post).

Like my Azure IoT The Things Industry connector I use Alastair Crabtrees’s LazyCache to store Azured IoT Hub DeviceClient instances.

NOTE: Swarm Space technical support clarified the parameter values required to get FromDevice and ToDevice messages using the Bumbleebee Hive API.

Swarm API Docs messages functionality

The Messages Get method has a lot of parameters for filtering and paging the response message lists. Many of the parameters have default values so can be null or left blank.

Swarm API Get User Message filters

I started off by seeing if I could duplicate the functionality of the user interface and get a list of all ToDevice and FromDevice messages.

Swarm Dashboard messages list

I first called the Messages Get method with the direction set to “fromdevice” (Odd this is a string rather than an enumeration) and the messages I had sent from my Sparkfun Satellite Transceiver Breakout – Swarm M138 were displayed.

Swarm API Docs displaying “fromdevice” messages

I then called the Messages Get method with the direction set to “all” and only the FromDevice messages were displayed which I wasn’t expecting.

Swarm API Docs displaying ToDevice and FromDevices messages

I then called the Messages Get method with the direction set to “FromDevice and no messages were displayed which I wasn’t expecting

Swarm API Docs displaying “todevice” messages

I then called the Message Get method with the messageId of a ToDevice message and the detailed message information was displayed.

Swarm API Docs displaying the details of a specific inbound message

For testing I configured 5 devices (a real device and the others simulated) in my Azure IoT Hub with the Swarm Device ID ued as the Azure IoT Hub device ID.

Devices configured in Azure IoT Hub

My console application calls the Swarm Bumblebee Hive API Login method, then uses Azure IoT Hub DeviceClient SendEventAsync upload device telemetry.

Nasty console application processing the three “fromdevice” messages which have not been acknowledged.

The console application stores the Swarm Hive API username, password and the Azure IoT Hub Device Connection string locally using the UserSecretsConfigurationExtension.

internal class Program
{
    private static string AzureIoTHubConnectionString = "";
    private readonly static IAppCache _DeviceClients = new CachingService();

    static async Task Main(string[] args)
    {
        Debug.WriteLine("devMobile.SwarmSpace.Hive.AzureIoTHubBasicClient starting");

        IConfiguration configuration = new ConfigurationBuilder()
            .SetBasePath(Directory.GetCurrentDirectory())
            .AddJsonFile("appsettings.json")
            .AddUserSecrets("b4073481-67e9-41bd-bf98-7d2029a0b391").Build();

        AzureIoTHubConnectionString = configuration.GetConnectionString("AzureIoTHub");

        using (HttpClient httpClient = new HttpClient())
        {
            BumblebeeHiveClient.Client client = new BumblebeeHiveClient.Client(httpClient);

            client.BaseUrl = configuration.GetRequiredSection("SwarmConnection").GetRequiredSection("BaseURL").Value;

            BumblebeeHiveClient.LoginForm loginForm = new BumblebeeHiveClient.LoginForm();

            loginForm.Username = configuration.GetRequiredSection("SwarmConnection").GetRequiredSection("UserName").Value;
            loginForm.Password = configuration.GetRequiredSection("SwarmConnection").GetRequiredSection("Password").Value;

            BumblebeeHiveClient.Response response = await client.PostLoginAsync(loginForm);

            Debug.WriteLine($"Token :{response.Token[..5]}.....{response.Token[^5..]}");

            string apiKey = "bearer " + response.Token;
            httpClient.DefaultRequestHeaders.Add("Authorization", apiKey);

            var devices = await client.GetDevicesAsync(null, null, null, null, null, null, null, null, null);

            foreach (BumblebeeHiveClient.Device device in devices)
            {
                Debug.WriteLine($" Id:{device.DeviceId} Name:{device.DeviceName} Type:{device.DeviceType} Organisation:{device.OrganizationId}");

                DeviceClient deviceClient = await _DeviceClients.GetOrAddAsync<DeviceClient>(device.DeviceId.ToString(), (ICacheEntry x) => IoTHubConnectAsync(device.DeviceId.ToString()), memoryCacheEntryOptions);
            }

            foreach (BumblebeeHiveClient.Device device in devices)
            {
                DeviceClient deviceClient = await _DeviceClients.GetAsync<DeviceClient>(device.DeviceId.ToString());

                var messages = await client.GetMessagesAsync(null, null, null, device.DeviceId.ToString(), null, null, null, null, null, null, "all", null, null);
                foreach (var message in messages)
                {
                    Debug.WriteLine($" PacketId:{message.PacketId} Status:{message.Status} Direction:{message.Direction} Length:{message.Len} Data: {BitConverter.ToString(message.Data)}");

                    JObject telemetryEvent = new JObject
                    {
                        { "DeviceID", device.DeviceId },
                        { "ReceivedAtUtc", DateTime.UtcNow.ToString("s", CultureInfo.InvariantCulture) },
                    };

                    telemetryEvent.Add("Payload",BitConverter.ToString(message.Data));

                    using (Message telemetryMessage = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(telemetryEvent))))
                    {
                        telemetryMessage.Properties.Add("iothub-creation-time-utc", message.HiveRxTime.ToString("s", CultureInfo.InvariantCulture));

                        await deviceClient.SendEventAsync(telemetryMessage);
                    };

                    //BumblebeeHiveClient.PacketPostReturn packetPostReturn = await client.AckRxMessageAsync(message.PacketId, null);
                }
            }

            foreach (BumblebeeHiveClient.Device device in devices)
            {
                DeviceClient deviceClient = await _DeviceClients.GetAsync<DeviceClient>(device.DeviceId.ToString());

                await deviceClient.CloseAsync();
            }
        }
    }

    private static async Task<DeviceClient> IoTHubConnectAsync(string deviceId)
    {
        DeviceClient deviceClient;

        deviceClient = DeviceClient.CreateFromConnectionString(AzureIoTHubConnectionString, deviceId, TransportSettings);

        await deviceClient.OpenAsync();

        return deviceClient;
    }

    private static readonly MemoryCacheEntryOptions memoryCacheEntryOptions = new MemoryCacheEntryOptions()
    {
        Priority = CacheItemPriority.NeverRemove
    };

    private static readonly ITransportSettings[] TransportSettings = new ITransportSettings[]
    {
        new AmqpTransportSettings(TransportType.Amqp_Tcp_Only)
        {
            AmqpConnectionPoolSettings = new AmqpConnectionPoolSettings()
            {
                Pooling = true,
            }
        }
    };
}

While testing I disabled the message RxAck functionality so I could repeatedly call the MessagesGet method so I didn’t have to send new messages and burn through my 50 free messages.

Azure IoT Explorer telemetry displaying the three messages processed by my console application.

.

Updated parameters based on feedback from Swarm technical support

Need to have status set to -1

Swarm Space – Bumblebee Hive Basic Emulator

One of the main problems building a Cloud Identity Translation Gateway (like my TTIV3AzureIoTConnector) is getting enough devices to make testing (esp. scalability) realistic. This is a problem because I have only got two devices, a Sparkfun Satellite Transceiver Breakout – Swarm M138 and a Swarm Asset Tracker. (Considering buying a Swarm Eval Kit)

Satellite Transceiver Breakout – Swarm M138
Swarm Asset Tracker

So, I can simulate lots of devices and test more complex configurations I have started build a Swarm Bumble Bee Hive emulator based on the API and Delivery-API OpenAPI files.

NSwagStudio configuration for generating ASP.NET Core web API

As well as generating clients NSwagStudio can also generate ASP.NET Core web APIs. To test my approach, I built the simplest possible client I could which calls the generated PostLoginAsync and GetDeviceCountAsync.

Swagger UI for NSwagStudio generated ASP.NET Core web API

Initially the BumblebeeHiveBasicClientConsole login method would fail with an HTTP 415 Unsupported Media Type error.

BumblebeeHiveBasicClientConsole application 415 Unsupported Media Type error

After some trial and error, I modified the HiveController.cs and HiveControllerImplementation.cs Login method signatures so the payload was “application/x-www-form-urlencoded” rather than “application/json” by changing FromBody to FromForm

Task<Response> IAuthController.PostLoginAsync([FromForm] LoginForm body)
{
     return Task.FromResult(new Response()
    {
        Token = Guid.NewGuid().ToString()
    });
}

Modifying code generated by a tool like NSwagStudio should be avoided but I couldn’t work out a simpler solution

/// <summary>
/// POST login
/// </summary>
/// <remarks>
/// &lt;p&gt;Use username and password to log in.&lt;/p&gt;&lt;p&gt;On success: returns status code 200. The response body is the JSON &lt;code&gt;{"token": "&amp;lt;token&amp;gt;"}&lt;/code&gt;, along with the header &lt;code&gt;Set-Cookie: JSESSIONID=&amp;lt;token&amp;gt;; Path=/; Secure; HttpOnly;&lt;/code&gt;. The tokens in the return value and the &lt;code&gt;Set-Cookie&lt;/code&gt; header are the same. The token is a long string of letters, numbers, and punctuation.&lt;/p&gt;&lt;p&gt;On failure: returns status code 401.&lt;/p&gt;&lt;p&gt;To make authenticated requests, there are two ways: &lt;ul&gt;&lt;li&gt;(Preferred) Use the token as a Bearer Authentication token by including the HTTP header &lt;code&gt;Authorization: Bearer &amp;lt;token&amp;gt;&lt;/code&gt; in further requests.&lt;/li&gt;&lt;li&gt;(Deprecated) Use the token as the JSESSIONID cookie in further requests.&lt;/li&gt;&lt;/ul&gt;&lt;/p&gt;
/// </remarks>
/// <returns>Login success</returns>
[Microsoft.AspNetCore.Mvc.HttpPost, Microsoft.AspNetCore.Mvc.Route("login")]
public System.Threading.Tasks.Task<Response> PostLogin([Microsoft.AspNetCore.Mvc.FromForm] LoginForm body)
{

    return _implementation.PostLoginAsync(body);
}

BumblebeeHiveBasicCLientConsole application calling the simulator
BumblebeeHiveBasicClientConsole application calling the production system

After some initial problems with content-types the Swarm Hive API (not tried the Delivery-API yet) appears to be documented and easy to use. Though, some of the variable type choices do seem a bit odd.

public virtual async System.Threading.Tasks.Task<string> GetDeviceCountAsync(int? devicetype, System.Threading.CancellationToken cancellationToken)

Swarm Space – Bumblebee Hive API Basic client

Back in July I purchased a Satellite Transceiver Breakout – Swarm M138 from SparkFun and it has been sitting on the shelf since then. I want to get telemetry from a sensor to an Azure IoT Hub or Azure IoT Central over a Swarm Space link for a project I am working on.

I’ll need to solder on some headers and cut a couple of tracks on the breakout board so my device (most probably a SparkFun – ESP32-S2 WROOM) can connect to the Swarm-M1138 modem. The NET nanoFramework team have an IoT.Device Swarm Tile NuGet package which I will use to interface the device to the modem.

I have started with a “nasty” Proof of Concept(PoC) to figure out how to connect to the Swarm Hive API.

The Swarm Hive API has been published with Swagger/OpenAPI which is really simple to use. I used NSwagStudio to generate a C# client to I didn’t have to “handcraft” one.

Initially the code would compile but I found a clue in a Github Issue from September 2017 which was to change the “Operation Generation Model” to SingleClientFromOperationId.(The setting is highlighted above).

static async Task Main(string[] args)
{
    using (HttpClient httpClient = new HttpClient())
    {
        BumblebeeHiveClient.Client client = new BumblebeeHiveClient.Client(httpClient);

        client.BaseUrl = "https://bumblebee.hive.swarm.space/hive/";

        BumblebeeHiveClient.LoginForm loginForm = new BumblebeeHiveClient.LoginForm();

        // https://bumblebee.hive.swarm.space/login/
        loginForm.Username = "...";
        loginForm.Password = "...";

        Console.WriteLine($"devMobile SwarmSpace Bumblebee Hive Console Client");
        Console.WriteLine("");

        Console.WriteLine($"Login POST");
        BumblebeeHiveClient.Response response = await client.PostLoginAsync(loginForm);

        Console.WriteLine($"Token :{response.Token[..5]}.....{response.Token[^5..]}");
        Console.WriteLine($"Press <enter> to continue");
        Console.ReadLine();

        string apiKey = "bearer " + response.Token;

        httpClient.DefaultRequestHeaders.Add("Authorization", apiKey);


        Console.WriteLine($"Device count GET");

        string count = await client.GetDeviceCountAsync(1);

        Console.WriteLine($"Device count :{count}");
        Console.WriteLine($"Press <enter> to continue");
        Console.ReadLine();

        Console.WriteLine($"Device(s) information GET");

        var devices = await client.GetDevicesAsync(1, null, null, null, null, null, null, null, null);

        foreach (var device in devices)
        {
            Console.WriteLine($" Id:{device.DeviceId} Name:{device.DeviceName} Type:{device.DeviceType} Organisation:{device.OrganizationId}");
        }

        Console.WriteLine($"Press <enter> to continue");
        Console.ReadLine();

        Console.WriteLine($"User Context GET");
        var userContext = await client.GetUserContextAsync();

        Console.WriteLine($" Id:{userContext.UserId} Name:{userContext.Username} Country:{userContext.Country}");

        Console.WriteLine("Additional properties");
        foreach ( var additionalProperty in userContext.AdditionalProperties)
        {
            Console.WriteLine($" Id:{additionalProperty.Key} Value:{additionalProperty.Value}");
        }

        Console.WriteLine($"Press <enter> to exit");
        Console.ReadLine();
    }
}

I tried a couple of ways to attach the Swarm Hive API authorisation token (returned by the Login method) to client requests. After a couple for failed attempts, I “realised” that adding the “Authorization” header to the HttpClient defaultRequestHeaders was by far the simplest approach.

My “nasty” console application calls the Login method, then requests the number of devices (I only have one), gets a list of the properties of all the devices(very short list) then gets the User Context and displays their ID, Name and Country.

libcamera-jpeg on Raspberry Pi OS Bullseye Duration

The image capture process was taking about 5 seconds which a bit longer than I was expecting.

libcamera-jpeg -o rotated.jpg --rotation 180

The libcamera-jpeg program has a lot of command line parameters.

pi@raspberrypi4a:~ $ libcamera-jpeg --help
Valid options are:
  -h [ --help ] [=arg(=1)] (=0)         Print this help message
  --version [=arg(=1)] (=0)             Displays the build version number
  -v [ --verbose ] [=arg(=1)] (=0)      Output extra debug and diagnostics
  -c [ --config ] [=arg(=config.txt)]   Read the options from a file. If no filename is specified, default to
                                        config.txt. In case of duplicate options, the ones provided on the command line
                                        will be used. Note that the config file must only contain the long form
                                        options.
  --info-text arg (=#%frame (%fps fps) exp %exp ag %ag dg %dg)
                                        Sets the information string on the titlebar. Available values:
                                        %frame (frame number)
                                        %fps (framerate)
                                        %exp (shutter speed)
                                        %ag (analogue gain)
                                        %dg (digital gain)
                                        %rg (red colour gain)
                                        %bg (blue colour gain)
                                        %focus (focus FoM value)
                                        %aelock (AE locked status)
  --width arg (=0)                      Set the output image width (0 = use default value)
  --height arg (=0)                     Set the output image height (0 = use default value)
  -t [ --timeout ] arg (=5000)          Time (in ms) for which program runs
  -o [ --output ] arg                   Set the output file name
  --post-process-file arg               Set the file name for configuring the post-processing
  --rawfull [=arg(=1)] (=0)             Force use of full resolution raw frames
  -n [ --nopreview ] [=arg(=1)] (=0)    Do not show a preview window
  -p [ --preview ] arg (=0,0,0,0)       Set the preview window dimensions, given as x,y,width,height e.g. 0,0,640,480
  -f [ --fullscreen ] [=arg(=1)] (=0)   Use a fullscreen preview window
  --qt-preview [=arg(=1)] (=0)          Use Qt-based preview window (WARNING: causes heavy CPU load, fullscreen not
                                        supported)
  --hflip [=arg(=1)] (=0)               Request a horizontal flip transform
  --vflip [=arg(=1)] (=0)               Request a vertical flip transform
  --rotation arg (=0)                   Request an image rotation, 0 or 180
  --roi arg (=0,0,0,0)                  Set region of interest (digital zoom) e.g. 0.25,0.25,0.5,0.5
  --shutter arg (=0)                    Set a fixed shutter speed
  --analoggain arg (=0)                 Set a fixed gain value (synonym for 'gain' option)
  --gain arg                            Set a fixed gain value
  --metering arg (=centre)              Set the metering mode (centre, spot, average, custom)
  --exposure arg (=normal)              Set the exposure mode (normal, sport)
  --ev arg (=0)                         Set the EV exposure compensation, where 0 = no change
  --awb arg (=auto)                     Set the AWB mode (auto, incandescent, tungsten, fluorescent, indoor, daylight,
                                        cloudy, custom)
  --awbgains arg (=0,0)                 Set explict red and blue gains (disable the automatic AWB algorithm)
  --flush [=arg(=1)] (=0)               Flush output data as soon as possible
  --wrap arg (=0)                       When writing multiple output files, reset the counter when it reaches this
                                        number
  --brightness arg (=0)                 Adjust the brightness of the output images, in the range -1.0 to 1.0
  --contrast arg (=1)                   Adjust the contrast of the output image, where 1.0 = normal contrast
  --saturation arg (=1)                 Adjust the colour saturation of the output, where 1.0 = normal and 0.0 =
                                        greyscale
  --sharpness arg (=1)                  Adjust the sharpness of the output image, where 1.0 = normal sharpening
  --framerate arg (=30)                 Set the fixed framerate for preview and video modes
  --denoise arg (=auto)                 Sets the Denoise operating mode: auto, off, cdn_off, cdn_fast, cdn_hq
  --viewfinder-width arg (=0)           Width of viewfinder frames from the camera (distinct from the preview window
                                        size
  --viewfinder-height arg (=0)          Height of viewfinder frames from the camera (distinct from the preview window
                                        size)
  --tuning-file arg (=-)                Name of camera tuning file to use, omit this option for libcamera default
                                        behaviour
  --lores-width arg (=0)                Width of low resolution frames (use 0 to omit low resolution stream
  --lores-height arg (=0)               Height of low resolution frames (use 0 to omit low resolution stream
  -q [ --quality ] arg (=93)            Set the JPEG quality parameter
  -x [ --exif ] arg                     Add these extra EXIF tags to the output file
  --timelapse arg (=0)                  Time interval (in ms) between timelapse captures
  --framestart arg (=0)                 Initial frame counter value for timelapse captures
  --datetime [=arg(=1)] (=0)            Use date format for output file names
  --timestamp [=arg(=1)] (=0)           Use system timestamps for output file names
  --restart arg (=0)                    Set JPEG restart interval
  -k [ --keypress ] [=arg(=1)] (=0)     Perform capture when ENTER pressed
  -s [ --signal ] [=arg(=1)] (=0)       Perform capture when signal received
  --thumb arg (=320:240:70)             Set thumbnail parameters as width:height:quality
  -e [ --encoding ] arg (=jpg)          Set the desired output encoding, either jpg, png, rgb, bmp or yuv420
  -r [ --raw ] [=arg(=1)] (=0)          Also save raw file in DNG format
  --latest arg                          Create a symbolic link with this name to most recent saved file
  --immediate [=arg(=1)] (=0)           Perform first capture immediately, with no preview phase
pi@raspberrypi4a:~ $

My libcamera-jpeg application is run “headless” so I tried turning off the image preview functionality.

libcamera-jpeg -o rotatednopreview.jpg --nopreview

When I ran libcamera-jpeg in a console windows or my application this didn’t appear to make any noticeable difference.

libcamera-jpeg run from the command line with –nopreview

libcamera-jpeg run by my application with –nopreview

I then had another look at the libcamera-jpeg command line parameters to see if any looked useful for reducing the time that it took to take a save an image and this one caught my attention.

I had assumed the delay was related to how long the preview window was displayed.

libcamera-jpeg run from the command line with –nopreview –t1

I modified the application (V5) then ran it from the command line and the time reduced to less than a second.

private static void ImageUpdateTimerCallback(object state)
{
	try
	{
		Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss} Image update start");

		// Just incase - stop code being called while photo already in progress
		if (_cameraBusy)
		{
			return;
		}

		Console.WriteLine($" {DateTime.UtcNow:yy-MM-dd HH:mm:ss} Image capture start");

		using (Process process = new Process())
		{
			process.StartInfo.FileName = @"libcamera-jpeg";
			// V1 it works
			//process.StartInfo.Arguments = $"-o {_applicationSettings.ImageFilenameLocal}";
			// V3a Image right way up
			//process.StartInfo.Arguments = $"-o {_applicationSettings.ImageFilenameLocal} --vflip --hflip";
			// V3b Image right way up
			//process.StartInfo.Arguments = $"-o {_applicationSettings.ImageFilenameLocal} --rotation 180";
			// V4 Image no preview
			//process.StartInfo.Arguments = $"-o {_applicationSettings.ImageFilenameLocal} --rotation 180 --nopreview";
			// V5 Image no preview, no timeout
			process.StartInfo.Arguments = $"-o {_applicationSettings.ImageFilenameLocal} --nopreview -t1 --rotation 180";
			//process.StartInfo.RedirectStandardOutput = true;
			// V2 No diagnostics
			process.StartInfo.RedirectStandardError = true;
			//process.StartInfo.UseShellExecute = false;
			//process.StartInfo.CreateNoWindow = true; 

			process.Start();

			if (!process.WaitForExit(10000) || (process.ExitCode != 0))
			{
				Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss} Image update failure {process.ExitCode}");
			}
		}

		Console.WriteLine($" {DateTime.UtcNow:yy-MM-dd HH:mm:ss} Image capture done");
	}
	catch (Exception ex)
	{
		Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss} Image update error {ex.Message}");
	}
	finally
	{
		_cameraBusy = false;
	}
}
libcamera-jpeg run by my application with –nopreview -t1

The image capture process now takes less that a second which is much better (but not a lot less than retrieving an image from one of my security cameras).