Chat Completion DTO Client Dog fooding

Posted on by

I have found “dogfooding” is one of the fastest ways to expose any rough edges. So, I got one my Mistral.AI Client applications to generate the code for a Mistral.AI Chat Completion client.

The output looked reasonable, so I copied and paste the code into an empty application.

Enter chat message: console application which calls he MistralAI Chat completion endpoint, api & model configuration loaded from appsetings, usersecrets for API-Key, should use typed DTOs, buffered API calls only (no streaming), httpClientFactory with resilence for http call. The application should wait for user to enter a prompt, if blank prompt program exits. If valid prompt request object generated, then text response displayed
# MistralAI Console Application with Resilient HTTP Calls

Here's a complete console application that meets your requirements. It uses:
- Configuration from appsettings.json
- User secrets for API key
- Typed DTOs
- HttpClientFactory with resilience
- Buffered API calls (no streaming)
- Interactive prompt input

## Project Structure

1. First, create a new console application:
```bash
dotnet new console -n MistralAIConsole
cd MistralAIConsole
```

2. Add required NuGet packages:
```bash
dotnet add package Microsoft.Extensions.Configuration
dotnet add package Microsoft.Extensions.Configuration.Binder
dotnet add package Microsoft.Extensions.Configuration.Json
dotnet add package Microsoft.Extensions.Configuration.UserSecrets
dotnet add package Microsoft.Extensions.DependencyInjection
dotnet add package Microsoft.Extensions.Http
dotnet add package Microsoft.Extensions.Http.Resilience
dotnet add package System.Text.Json
```

## Implementation

### 1. Configuration Classes

Create `Models/Configuration.cs`:
```csharp
namespace MistralAIConsole.Models;

public class MistralAIConfiguration
{
    public string ApiUrl { get; set; } = string.Empty;
    public string Model { get; set; } = string.Empty;
    public int MaxTokens { get; set; } = 1024;
    public double Temperature { get; set; } = 0.7;
}

public class AppConfiguration
{
    public MistralAIConfiguration MistralAI { get; set; } = new();
}
```

### 2. DTO Classes

Create `Models/RequestResponse.cs`:
```csharp
using System.Text.Json.Serialization;

namespace MistralAIConsole.Models;

public class ChatCompletionRequest
{
    [JsonPropertyName("model")]
    public string Model { get; set; } = string.Empty;

    [JsonPropertyName("messages")]
    public List<ChatMessage> Messages { get; set; } = new();

    [JsonPropertyName("max_tokens")]
    public int MaxTokens { get; set; } = 1024;

    [JsonPropertyName("temperature")]
    public double Temperature { get; set; } = 0.7;
}

public class ChatMessage
{
    [JsonPropertyName("role")]
    public string Role { get; set; } = "user";

    [JsonPropertyName("content")]
    public string Content { get; set; } = string.Empty;
}

public class ChatCompletionResponse
{
    [JsonPropertyName("id")]
    public string Id { get; set; } = string.Empty;

    [JsonPropertyName("object")]
    public string Object { get; set; } = string.Empty;

    [JsonPropertyName("created")]
    public long Created { get; set; }

    [JsonPropertyName("model")]
    public string Model { get; set; } = string.Empty;

    [JsonPropertyName("choices")]
    public List<ChatChoice> Choices { get; set; } = new();

    [JsonPropertyName("usage")]
    public UsageInfo Usage { get; set; } = new();
}

public class ChatChoice
{
    [JsonPropertyName("index")]
    public int Index { get; set; }

    [JsonPropertyName("message")]
    public ChatMessage Message { get; set; } = new();

    [JsonPropertyName("finish_reason")]
    public string FinishReason { get; set; } = string.Empty;
}

public class UsageInfo
{
    [JsonPropertyName("prompt_tokens")]
    public int PromptTokens { get; set; }

    [JsonPropertyName("completion_tokens")]
    public int CompletionTokens { get; set; }

    [JsonPropertyName("total_tokens")]
    public int TotalTokens { get; set; }
}
```

### 3. Service Interface and Implementation

Create `Services/IChatService.cs`:
```csharp
using MistralAIConsole.Models;

namespace MistralAIConsole.Services;

public interface IChatService
{
    Task<ChatCompletionResponse> GetChatCompletionAsync(ChatCompletionRequest request);
}
```

Create `Services/ChatService.cs`:
```csharp
using System.Net.Http.Json;
using System.Text.Json;
using MistralAIConsole.Models;
using Microsoft.Extensions.Options;

namespace MistralAIConsole.Services;

public class ChatService : IChatService
{
    private readonly HttpClient _httpClient;
    private readonly MistralAIConfiguration _config;

    public ChatService(HttpClient httpClient, IOptions<MistralAIConfiguration> config)
    {
        _httpClient = httpClient;
        _config = config.Value;
    }

    public async Task<ChatCompletionResponse> GetChatCompletionAsync(ChatCompletionRequest request)
    {
        var response = await _httpClient.PostAsJsonAsync("chat/completions", request);
        response.EnsureSuccessStatusCode();

        var content = await response.Content.ReadAsStringAsync();
        return JsonSerializer.Deserialize<ChatCompletionResponse>(content) ??
               throw new InvalidOperationException("Failed to deserialize response");
    }
}
```

### 4. Program.cs

```csharp
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Http.Resilience;
using Microsoft.Extensions.Http;
using MistralAIConsole.Models;
using MistralAIConsole.Services;
using Polly;

var configuration = new ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
    .AddUserSecrets<Program>()
    .Build();

var services = new ServiceCollection();

services.Configure<AppConfiguration>(configuration);
services.Configure<MistralAIConfiguration>(configuration.GetSection("MistralAI"));

services.AddHttpClient<IChatService, ChatService>(client =>
    {
        var mistralConfig = configuration.GetSection("MistralAI").Get<MistralAIConfiguration>();
        client.BaseAddress = new Uri(mistralConfig!.ApiUrl);
        client.DefaultRequestHeaders.Add("Authorization", $"Bearer {configuration["MistralAI:ApiKey"]}");
    })
    .AddStandardResilienceHandler()
    .Configure<StandardResilienceOptions>(options =>
    {
        options.Retry.MaxRetryAttempts = 3;
        options.Retry.Delay = TimeSpan.FromSeconds(1);
        options.Retry.BackoffType = DelayBackoffType.Exponential;
        options.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(10);
        options.CircuitBreaker.FailureRatio = 0.5;
        options.CircuitBreaker.MinimumThroughput = 5;
        options.CircuitBreaker.BreakDuration = TimeSpan.FromSeconds(30);
    });

var serviceProvider = services.BuildServiceProvider();

Console.WriteLine("MistralAI Console Client");
Console.WriteLine("Enter your prompt (or press Enter to exit):");

while (true)
{
    Console.Write("> ");
    var prompt = Console.ReadLine();

    if (string.IsNullOrWhiteSpace(prompt))
    {
        Console.WriteLine("Exiting...");
        break;
    }

    try
    {
        var chatService = serviceProvider.GetRequiredService<IChatService>();
        var config = serviceProvider.GetRequiredService<IOptions<MistralAIConfiguration>>().Value;

        var request = new ChatCompletionRequest
        {
            Model = config.Model,
            MaxTokens = config.MaxTokens,
            Temperature = config.Temperature,
            Messages = new List<ChatMessage>
            {
                new() { Role = "user", Content = prompt }
            }
        };

        Console.WriteLine("Sending request to MistralAI...");
        var response = await chatService.GetChatCompletionAsync(request);

        Console.WriteLine("\nResponse:");
        Console.WriteLine(response.Choices.FirstOrDefault()?.Message?.Content ?? "No response content");
        Console.WriteLine($"\nTokens used: {response.Usage.TotalTokens}");
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Error: {ex.Message}");
    }

    Console.WriteLine();
}

await Task.CompletedTask;
```

### 5. Configuration Files

Create `appsettings.json`:
```json
{
  "MistralAI": {
    "ApiUrl": "https://api.mistral.ai/v1/",
    "Model": "mistral-tiny",
    "MaxTokens": 1024,
    "Temperature": 0.7
  }
}
```

Initialize user secrets:
```bash
dotnet user-secrets init
dotnet user-secrets set "MistralAI:ApiKey" "your-api-key-here"
```

## How to Use

1. Replace `your-api-key-here` in the user secrets with your actual MistralAI API key
2. Run the application:
```bash
dotnet run
```
3. Enter your prompt when prompted
4. The application will display the response from MistralAI
5. Press Enter without typing anything to exit

## Features

- Resilient HTTP calls with retry and circuit breaker patterns
- Configuration loaded from appsettings.json and user secrets
- Typed DTOs for request/response
- Buffered API calls (no streaming)
- Interactive prompt input
- Proper error handling

The application will automatically exit when an empty prompt is entered.

Prompt tokens: 87
Completion tokens: 1948
Total tokens: 2035

Enter chat message:

It was looking really promising until I tried to compile the code. The resilience handler configuration code was broken so I tried Github Copilot but the suggested fix was overkill with lots of “magic numbers”. I then manually fixed the issue by deleting some code (less is more).

var services = new ServiceCollection();

services.Configure<AppConfiguration>(configuration);
services.Configure<MistralAIConfiguration>(configuration.GetSection("MistralAI"));

/* Original code with broken resilience handler configuration 
services.AddHttpClient<IChatService, ChatService>(client =>
    {
        var mistralConfig = configuration.GetSection("MistralAI").Get<MistralAIConfiguration>();
        client.BaseAddress = new Uri(mistralConfig!.ApiUrl);
        client.DefaultRequestHeaders.Add("Authorization", $"Bearer {configuration["MistralAI:ApiKey"]}");
    })
    .AddStandardResilienceHandler()
    .Configure<StandardResilienceOptions>(options =>
    {
        options.Retry.MaxRetryAttempts = 3;
        options.Retry.Delay = TimeSpan.FromSeconds(1);
        options.Retry.BackoffType = DelayBackoffType.Exponential;
        options.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(10);
        options.CircuitBreaker.FailureRatio = 0.5;
        options.CircuitBreaker.MinimumThroughput = 5;
        options.CircuitBreaker.BreakDuration = TimeSpan.FromSeconds(30);
    });
*/

// Smallest fix for the resilience handler
services.AddHttpClient<IChatService, ChatService>(client =>
{
   var mistralConfig = configuration.GetSection("MistralAI").Get<MistralAIConfiguration>();
   client.BaseAddress = new Uri(mistralConfig!.ApiUrl);
   client.DefaultRequestHeaders.Add("Authorization", $"Bearer {configuration["MistralAI:ApiKey"]}");
}).AddStandardResilienceHandler();

/* Copilot's fix for the resilience handler, way more complex the required for PoC
services.AddHttpClient<IChatService, ChatService>(client =>
{
   var mistralConfig = configuration.GetSection("MistralAI").Get<MistralAIConfiguration>();
   client.BaseAddress = new Uri(mistralConfig!.ApiUrl);
   client.DefaultRequestHeaders.Add("Authorization", $"Bearer {configuration["MistralAI:ApiKey"]}");
})
.AddStandardResilienceHandler()
.Configure(options =>
{
   // Retry
   options.Retry.MaxRetryAttempts = 3;
   options.Retry.Delay            = TimeSpan.FromSeconds(1);
   options.Retry.BackoffType      = DelayBackoffType.Exponential;

   // Circuit breaker — SamplingDuration must be >= 2 * AttemptTimeout.Timeout
   options.CircuitBreaker.SamplingDuration  = TimeSpan.FromSeconds(30);   // was 10 → invalid
   options.CircuitBreaker.FailureRatio      = 0.5;
   options.CircuitBreaker.MinimumThroughput = 5;
   options.CircuitBreaker.BreakDuration     = TimeSpan.FromSeconds(30);

   // Give the retry budget room:
   //   MaxRetryAttempts * (Retry.Delay + AttemptTimeout.Timeout) <= TotalRequestTimeout.Timeout
   //   3 * (1s + 10s) = 33s  →  must raise TotalRequestTimeout above 33s
   options.AttemptTimeout.Timeout      = TimeSpan.FromSeconds(10);
   options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(60);
});
*/

var serviceProvider = services.BuildServiceProvider();

The Anthropic Claude review exported as markdown hence the odd formatting. I couldn’t export Copilot Chat

Code Review: TypedDTOsGeneratedClient

Reviewed files: Program.cs, Configuration.cs, RequestResponse.cs, Services/ChatService.cs, Services/IChatService.cs, TypedDTOsGeneratedClient.csproj, appsettings.json

Bugs / correctness

  1. Swallowed error detail on failed HTTP calls (Services/ChatService.cs:22) response.EnsureSuccessStatusCode() throws before the response body is read. Mistral’s API returns useful error detail (e.g. auth failures, rate limits, invalid model) in the JSON body, but that’s discarded — the user just sees "Response status code does not indicate success: 401 (Unauthorized)." in the console. Read the body first and include it in a thrown exception when the status isn’t successful.

  2. serviceProvider is never disposed (Program.cs:78) BuildServiceProvider() returns an IDisposable (it owns the HttpClient/IHttpClientFactory and resilience handlers). Should be using var serviceProvider = ....

Dead code / cleanup

  1. ~55 lines of commented-out code (Program.cs:21-76) Three abandoned attempts at configuring AddStandardResilienceHandler (original, “smallest fix”, “Copilot’s fix”), left in as block comments. This is exactly what git history is for — worth deleting now that the active version works, so the file isn’t carrying a debugging diary.

  2. AppConfiguration is registered but never consumed (Program.cs:18, Configuration.cs:11-14) services.Configure(configuration) binds the whole root config, but nothing ever injects IOptions; only IOptions is used. Either use it or drop the registration and the class.

Design inconsistency

  1. ApiKey isn’t part of MistralAIConfiguration (Configuration.cs:3-9 vs Program.cs:26,46) ApiUrl, Model, MaxTokens, Temperature are all strongly-typed, but the API key is read via the raw string indexer configuration["MistralAI:ApiKey"]. If deliberate (to avoid the secret riding along on the options object), a one-line comment would save the next reader from wondering; otherwise it’s a gap in an otherwise “Typed DTOs” project.

Minor / style

  1. ChatService.GetChatCompletionAsync manually does ReadAsStringAsync + JsonSerializer.Deserialize (ChatService.cs:24-25) where response.Content.ReadFromJsonAsync() would do the same in one call and reuse the DI-configured serializer options.
  2. No CancellationToken anywhere in the async chain (IChatService, ChatService, Program.cs’s loop) — fine for a console PoC, but worth noting if this client graduates beyond a demo.
  3. mistralConfig!.ApiUrl (Program.cs:45) uses null-forgiving on a config section that could legitimately be missing/misspelled — would NRE with no context if the MistralAI section is absent from appsettings.json.

Security notes

Nothing security-sensitive stood out — the API key correctly comes from user secrets, not appsettings.json (confirmed appsettings.json has no ApiKey field), and it’s not logged anywhere.

“Dogfooding" my client forced me to revisit the DTOs (especially the polymorphic ones), refine the resilience strategy, and debug the implementation of the prompt loop.

Overall, for a PoC the code and review results were surprisingly good. I have found “teaming” using Mistral Codestral, Anthropic Claude and Github Copilot to generate and then review each other’s code.

Mistral Steaming Chat Completions

Posted on by

This sample demonstrates how to consume Mistral’s streaming chat completion endpoint using HttpClient and Server-Sent Events (SSE). The implementation streams tokens to the console as they arrive, supports cancellation with Ctrl+C, and captures token usage information. A CancellationTokenSource is used to support user-initiated cancellation. Pressing Ctrl+C cancels the active request without terminating the application, allowing the user to continue interacting with the chat client.

Rather than waiting for the complete response, the request is sent with Stream = true and the response is processed as an SSE stream. This provides a significantly better user experience because generated tokens are displayed as soon as they are received. Each chunk contains a delta payload that may include newly generated content. 

// Ctrl+C cancels the current request and returns to the prompt; second Ctrl+C exits the process.
using var cts = new CancellationTokenSource();
Console.CancelKeyPress += (_, e) =>
{
   if (!cts.IsCancellationRequested)
   {
      e.Cancel = true;
      cts.Cancel();
   }
};

while (!cts.IsCancellationRequested)
{
   Console.Write("Enter chat message: ");
   var content = Console.ReadLine();
   if (string.IsNullOrWhiteSpace(content)) break;

   var request = new ChatStreamingCompletionRequest
   {
      Model = settings.ModelName,
      Messages =
      [
         new ChatMessage { Role = "user", Content = content }
      ],
      Stream = true,
   };

   try
   {
      using var httpRequest = new HttpRequestMessage(HttpMethod.Post, "chat/completions")
      {
         Content = JsonContent.Create(request, options: jsonSerializerOptions)
      };

      using var httpResponse = await httpClient.SendAsync(httpRequest, HttpCompletionOption.ResponseHeadersRead, cts.Token);
      if (!httpResponse.IsSuccessStatusCode)
      {
         var body = await httpResponse.Content.ReadAsStringAsync(cts.Token);
         throw new HttpRequestException($"Mistral API {(int)httpResponse.StatusCode}: {body}", null, httpResponse.StatusCode);
      }

      await using var stream = await httpResponse.Content.ReadAsStreamAsync(cts.Token);
      using var reader = new StreamReader(stream);

      TokenUsage? finalUsage = null;
      string? finishReason = null;

      string? line;
      // Mistral emits single-line `data:` chunks, so per-line dispatch is sufficient (no multi-line SSE concatenation required).
      while ((line = await reader.ReadLineAsync(cts.Token)) is not null)
      {
         // SSE framing: blank line separates events, lines without "data:" are ignored (e.g. ":" keep-alive comments, "event:" / "id:" / "retry:" fields).
         if (string.IsNullOrEmpty(line)) continue;
         if (!line.StartsWith("data:", StringComparison.Ordinal)) continue;

         var payload = line.AsSpan(5).TrimStart();
         if (payload.SequenceEqual("[DONE]")) break;

         var chunk = JsonSerializer.Deserialize<ChatCompletionChunk>(payload, jsonSerializerOptions);
         if (chunk is null) continue;

         foreach (var choice in chunk.Choices)
         {
            if (!string.IsNullOrEmpty(choice.Delta.Content))
            {
               Console.Write(choice.Delta.Content);
            }
            if (choice.FinishReason is not null)
            {
               finishReason = choice.FinishReason;
            }
         }

         if (chunk.Usage is not null)
         {
            finalUsage = chunk.Usage;
         }
      }

      Console.WriteLine();
      Console.WriteLine();
      if (finalUsage is not null)
      {
         Console.WriteLine($"Prompt tokens: {finalUsage.PromptTokens}");
         Console.WriteLine($"Completion tokens: {finalUsage.CompletionTokens}");
         Console.WriteLine($"Total tokens: {finalUsage.TotalTokens}");
      }
      if (finishReason is not null)
      {
         Console.WriteLine($"Finish reason: {finishReason}");
      }
      Console.WriteLine();
   }
   catch (OperationCanceledException) when (cts.IsCancellationRequested)
   {
      Console.WriteLine();
      Console.WriteLine("Request cancelled.");
      // Reset the CTS so the next prompt iteration is cancellable again.
      cts.TryReset();
   }
   catch (HttpRequestException ex)
   {
      Console.WriteLine($"Request failed: {(int?)ex.StatusCode ?? 0} {ex.Message}");
   }
   catch (TaskCanceledException)
   {
      Console.WriteLine("Request timed out.");
   }
   catch (JsonException ex)
   {
      Console.WriteLine($"Failed to parse response: {ex.Message}");
   }
   catch (IOException ex)
   {
      Console.WriteLine($"Stream error: {ex.Message}");
   }
   catch (Exception ex)
   {
      Console.WriteLine($"Unexpected error: {ex.GetType().Name}: {ex.Message}");
   }
}

Uses HttpCompletionOption.ResponseHeadersRead to begin processing data as soon as headers are received.

public sealed class ChatStreamingCompletionRequest
{
   [JsonPropertyName("model")] public required string Model { get; init; }
   [JsonPropertyName("messages")] public required List<ChatMessage> Messages { get; init; }
   [JsonPropertyName("temperature")] public double? Temperature { get; init; }
   [JsonPropertyName("max_tokens")] public int? MaxTokens { get; init; }
   [JsonPropertyName("stream")] public bool? Stream { get; init; } 
   [JsonPropertyName("response_format")] public ResponseFormat? ResponseFormat { get; init; }
}

public sealed class ChatMessage
{
   [JsonPropertyName("role")] public required string Role { get; init; }
   [JsonPropertyName("content")] public required string Content { get; init; }
}

public sealed class ResponseFormat
{
   [JsonPropertyName("type")] public required string Type { get; init; }
}

public sealed class ChatCompletionChunk
{
   [JsonPropertyName("id")] public required string Id { get; init; }
   [JsonPropertyName("choices")] public required List<ChatCompletionDelta> Choices { get; init; }
   [JsonPropertyName("usage")] public TokenUsage? Usage { get; init; }
}

public sealed class ChatCompletionDelta
{
   [JsonPropertyName("index")] public int Index { get; init; }
   [JsonPropertyName("delta")] public required ChatMessageDelta Delta { get; init; }
   [JsonPropertyName("finish_reason")] public string? FinishReason { get; init; }
}

public sealed class ChatMessageDelta
{
   [JsonPropertyName("role")] public string? Role { get; init; }
   [JsonPropertyName("content")] public string? Content { get; init; }
}

public sealed class TokenUsage
{
   [JsonPropertyName("prompt_tokens")] public int? PromptTokens { get; init; }
   [JsonPropertyName("completion_tokens")] public int? CompletionTokens { get; init; }
   [JsonPropertyName("total_tokens")] public int? TotalTokens { get; init; }
}

When available, token usage statistics are captured from the final response chunk and displayed after generation completes.

Figured I might as well get Anthropic’s Claude to review my code.

Code Review: MistralBasicStreamingCLI

Scope: files in MistralBasicStreamingCLI/ only (Program.cs, Model.cs, ApplicationSettings.cs, appsettings.json, .csproj).

Summary

Small, single-file-ish PoC CLI that streams Mistral chat completions over SSE. Overall solid for its stated purpose (the file header calls it “horrible” — it’s actually reasonably clean). Config/secrets handling, cancellation plumbing, and exception granularity are all good practice. Findings below are ranked roughly by severity.

Findings

1. Ctrl+C behavior contradicts its own comment (Medium)

Program.cs:43 says:

> Ctrl+C cancels the current request and returns to the prompt; second Ctrl+C exits the process.

That holds only if Ctrl+C is pressed while a request is in flight — in that case the catch (OperationCanceledException) block runs and calls cts.TryReset() (line 141), so the loop continues.

But if Ctrl+C is pressed while sitting at the Console.ReadLine() prompt (before any request is sent), cts.Cancel() fires, Console.ReadLine() returns (typically null), and content fails the IsNullOrWhiteSpace check on line 58, so the loop hits break directly — no TryReset() call, no “second Ctrl+C” needed. The program exits on the first Ctrl+C in this case.

Net effect: whether Ctrl+C exits immediately or returns you to the prompt depends on exact timing (mid-request vs. at-prompt), which will look like inconsistent/buggy behavior to a user. If the two-stage cancel is intended everywhere, the prompt-read would need to be cancellable too (e.g. via a Task.Run wrapping Console.ReadLine() racing the token, or Console.KeyAvailable polling).

2. Streaming Usage block may never populate (Worth verifying)

Program.cs:116-119 and 126-128 only print token counts if chunk.Usage is not null. For OpenAI-compatible streaming APIs, the final chunk generally only carries usage when the request explicitly opts in (OpenAI requires "stream_options": {"include_usage": true}). ChatStreamingCompletionRequest (Model.cs:5-13) has no StreamOptions/equivalent property, and the request never sends one.

If Mistral’s API follows the same opt-in convention, finalUsage will always stay null and the “Prompt tokens / Completion tokens / Total tokens” lines will silently never print. If Mistral always includes usage on the last chunk regardless (some of their docs suggest this), then it’s fine as-is. Worth a quick manual test against the real API to confirm which is true — if it’s the former, add a stream_options field to the request model.

3. Redundant double-dispose of the response stream (Low)

await using var stream = await httpResponse.Content.ReadAsStreamAsync(cts.Token);
using var reader = new StreamReader(stream);

reader’s (synchronous) Dispose() runs first at scope exit and disposes the underlying stream (default leaveOpen: false). The outer await using then calls DisposeAsync() on the same, already-disposed stream. This is harmless (Stream.Dispose/DisposeAsync are idempotent) but is dead code — either drop await using in favor of a plain using, or construct StreamReader with leaveOpen: true and let the outer await using be the one real disposal.

4. No conversation history — every turn is stateless (Design note)

Each loop iteration sends a single ChatMessage { Role = "user", Content = content } (Program.cs:60-68) with no prior turns or system prompt included. So the “chat loop” is really a sequence of independent one-shot completions, not a multi-turn conversation. Likely intentional for a minimal PoC, but worth calling out in case a real chat history was expected.

5. Unused request fields (Low / cosmetic)

Temperature, MaxTokens, and ResponseFormat on ChatStreamingCompletionRequest (Model.cs:9,10,12) are never set anywhere — every request sends null for all three (and they’re omitted from the JSON payload thanks to WhenWritingNull). Not a bug, just currently-dead surface area on the model; fine to leave if they’re there for future use, otherwise consider trimming until needed.

6. appsettings.json ships an empty ApiKey (Informational, not a defect)

appsettings.json:3 has "ApiKey": "", with the real key presumably supplied via dotnet user-secrets (UserSecretsId is wired up in the .csproj). This is the correct pattern — flagging only so it stays that way and nobody “temporarily” pastes a real key into this file for a quick test and commits it.

Positives worth keeping

  • Secrets handling: AddUserSecrets() + empty placeholder in appsettings.json is the right pattern for a local PoC.
  • Startup validation: ArgumentException.ThrowIfNullOrWhiteSpace on required settings fails fast with a clear message instead of a confusing null-ref later.
  • HttpClient.Timeout = Timeout.InfiniteTimeSpan paired with a CancellationToken is the correct way to handle long-lived SSE connections — avoids the common bug of a wall-clock timeout killing a slow-but-healthy stream.
  • BaseAddress/relative-URI combination (https://api.mistral.ai/v1/ + chat/completions) is done correctly (trailing slash on base, no leading slash on relative) — a very common source of bugs elsewhere.
  • SSE line parsing correctly ignores blank lines and non-data: fields (comments, event:, id:, retry:), and checks for [DONE] before attempting to parse JSON.
  • Exception granularity: catching OperationCanceledException, HttpRequestException, TaskCanceledException, JsonException, IOException separately before a generic fallback gives useful, distinct messages per failure mode rather than one catch-all.

Suggested priority

  1. Decide/fix the Ctrl+C-at-prompt vs. Ctrl+C-mid-request inconsistency (#1) if consistent behavior matters for this PoC’s usability.
  2. Confirm whether Usage actually arrives from Mistral’s streaming endpoint without stream_options (#2) — quick manual test settles it.
  3. Everything else is cosmetic/cleanup, safe to defer.

Using strongly typed request and response Data Transfer Objects(DTOs) improved maintainability but without manual “tweaking” a request/response could fail.(esp. required & nullable). This implementation assumes that the Mistral Chat Completion API returns single-line JavaScript Object Notation(JSON) payloads in each SSE data: event, which simplified the parser implementation.

Mistral Chat Completions

Posted on by

Building a lightweight Codestral Chat Completion Command-Line Interface(CLI) is one of the fastest ways to understand how modern Large Language Model(LLM) Application Programming Interfaces (API) work in the real world. While everyone is talking about “agentic” programming (I now spend a lot of time reviewing code, as more agents = more reviews) and “token maxing” (difficult conversations with finance) this series of posts is about the plumbing.

I’m starting from the bottom and working my way up the stack. A raw Hypertext Transfer Protocol(HTTP) contract: an HTTP POST, a model name, a messages array, and a response object you have to parse yourself. With an HTTP proxy (Telerik Fiddler) I confirmed the Mistral Chat API endpoint Uniform Resource Locator(URL) and my API Key worked.

Before writing or generating typed Data Transfer Objects(DTO), or any kind of strongly‑typed client, streaming the response JSON into a jsonDocument was a good way to visualise the shape of the responses. I could enumerate properties, check for missing or inconsistent fields, validate casing, and confirm whether optional objects appear only in certain scenarios. Any undocumented polymorphic shapes, and response constructs can be impossible to model cleanly, and “hand-rolled” serialisation can be fragile.

//...
Console.Write("Enter chat message: ");
var prompt = Console.ReadLine();

// Anonymous type for the request body which feels bit "hinky" but, it works and is concise. Alternatively, could define a class for request body for better type safety and maintainability.
var requestObject = new
{
   model = settings.ModelName,
   messages = new[]
   {
      new { role = "user", content = prompt }
   }
};

// Alternatively, a JsonObject and JsonArray for more control over the JSON structure
var requestJson = new JsonObject()
{
   ["model"] = settings.ModelName,
   ["messages"] = new JsonArray
   {
      new JsonObject
      {
         ["role"] = "user",
         ["content"] = prompt
      }
   }
};

// Create HttpClient with required headers. Note that HttpClient should ideally be reused, but for simplicity we're creating a new instance here.
HttpClient httpClient = new()
{
   DefaultRequestHeaders =
   {
      Accept = { new MediaTypeWithQualityHeaderValue("application/json") },
      Authorization = new AuthenticationHeaderValue("Bearer", settings.ApiKey)
   },
   BaseAddress = new Uri(settings.BaseUrl)
};

using var httpResponse = await httpClient.PostAsync("chat/completions", new StringContent(JsonSerializer.Serialize(requestObject), Encoding.UTF8, "application/json"));

httpResponse.EnsureSuccessStatusCode();

using var stream = await httpResponse.Content.ReadAsStreamAsync();
using var responseDocument = await JsonDocument.ParseAsync(stream);

var content = responseDocument.RootElement.GetProperty("choices")[0].GetProperty("message").GetProperty("content").GetString();

Console.WriteLine(content);
Console.WriteLine();

var usage = responseDocument.RootElement.GetProperty("usage");
Console.WriteLine($"Prompt tokens: {usage.GetProperty("prompt_tokens").GetInt32()}");
Console.WriteLine($"Completion tokens: {usage.GetProperty("completion_tokens").GetInt32()}");
Console.WriteLine($"Total tokens: {usage.GetProperty("total_tokens").GetInt32()}");
Console.WriteLine();

Console.WriteLine("Press <Enter> to exit...");
Console.ReadLine();

Reading the response string and parsing it with a jsonDocument is straightforward but can be inefficient for large responses because it loads the entire response into memory.

I built the typed interface by inspecting real JSON responses especially structures like polymorphic message, and cross‑checking them against the Mistral API docs. This highlighted which fields are genuinely optional, which only appear for tool calls, and which vary by finish reason. It also highlighted that I would have to introduce polymorphic message classes so the interface can cleanly represent text messages, tool‑call messages, and whatever new variants the API adds later.

// Create HttpClient with required headers. Note that HttpClient should ideally be reused, but for simplicity we're creating a new instance here..
using HttpClient httpClient = new()
{
   DefaultRequestHeaders =
   {
      Accept = { new MediaTypeWithQualityHeaderValue("application/json") },
      Authorization = new AuthenticationHeaderValue("Bearer", settings.ApiKey)
   },
   BaseAddress = new Uri(settings.BaseUrl)
};

var jsonSerializerOptions = new JsonSerializerOptions()
{
   // PropertyNamingPolicy removed - [JsonPropertyName] attributes on model handle wire names
   WriteIndented           = false,
   DefaultIgnoreCondition  = JsonIgnoreCondition.WhenWritingNull,
   AllowTrailingCommas     = false,
   ReadCommentHandling     = JsonCommentHandling.Disallow,
   UnmappedMemberHandling  = JsonUnmappedMemberHandling.Skip,
};

Console.Write("Enter chat message: ");
var content = Console.ReadLine();

while (!string.IsNullOrWhiteSpace(content))
{
   var request = new ChatCompletionRequest
   {
      Model = settings.ModelName,
      Messages =
      [
         new ChatMessage { Role = "user", Content = content }
      ],
   };

   try
   {
      using var httpResponse = await httpClient.PostAsJsonAsync("chat/completions", request, jsonSerializerOptions);
      httpResponse.EnsureSuccessStatusCode();

      ChatCompletionResponse? chatCompletionResponse = await httpResponse.Content.ReadFromJsonAsync<ChatCompletionResponse>(jsonSerializerOptions);

      if (chatCompletionResponse != null)
      {
         foreach (var choice in chatCompletionResponse.Choices)
         {
            Console.WriteLine(choice.Message.Content);
         }

         Console.WriteLine();
         if (chatCompletionResponse.Usage != null)
         {
            Console.WriteLine($"Prompt tokens: {chatCompletionResponse.Usage.PromptTokens}");
            Console.WriteLine($"Completion tokens: {chatCompletionResponse.Usage.CompletionTokens}");
            Console.WriteLine($"Total tokens: {chatCompletionResponse.Usage.TotalTokens}");
         }
         Console.WriteLine();
      }
   }
   catch (HttpRequestException ex)
   {
      Console.WriteLine($"Request failed: {(int?)ex.StatusCode} {ex.Message}");
   }
   catch (TaskCanceledException)
   {
      Console.WriteLine("Request timed out.");
   }
   catch (JsonException ex)
   {
      Console.WriteLine($"Failed to parse response: {ex.Message}");
   }

   Console.Write("Enter chat message: ");
   content = Console.ReadLine();
}

public sealed class ChatCompletionRequest
{
   [JsonPropertyName("model")] public required string Model { get; init; }
   [JsonPropertyName("messages")] public required List<ChatMessage> Messages { get; init; }
   [JsonPropertyName("temperature")] public double? Temperature { get; init; }
   [JsonPropertyName("max_tokens")] public int? MaxTokens { get; init; }
   [JsonPropertyName("stream")] public bool? Stream { get; init; }
   [JsonPropertyName("response_format")] public ResponseFormat? ResponseFormat { get; init; }
}

public sealed class ChatMessage
{
   [JsonPropertyName("role")] public required string Role { get; init; }
   [JsonPropertyName("content")] public required string Content { get; init; }
}

public sealed class ResponseFormat
{
   [JsonPropertyName("type")] public required string Type { get; init; }
}

public sealed class ChatCompletionResponse
{
   [JsonPropertyName("id")] public required string Id { get; init; }
   [JsonPropertyName("choices")] public required List<ChatCompletionChoice> Choices { get; init; }
   [JsonPropertyName("usage")] public TokenUsage? Usage { get; init; }
}

public sealed class TokenUsage
{
   [JsonPropertyName("prompt_tokens")] public int? PromptTokens { get; init; }
   [JsonPropertyName("completion_tokens")] public int? CompletionTokens { get; init; }
   [JsonPropertyName("total_tokens")] public int? TotalTokens { get; init; }
}

public sealed class ChatCompletionChoice
{
   [JsonPropertyName("index")] public int Index { get; init; }
   [JsonPropertyName("message")] public required ChatMessage Message { get; init; }
   [JsonPropertyName("finish_reason")] public string? FinishReason { get; init; }
}

I had to capture the application output in two screenshots as the response text was longer.

The non-deterministic nature of LLMs resulted in different response messages, with the longest one consuming significantly more tokens, 530 vs. 799 (future posts will cover the use of Random_seed)

Real-time IoT Change Point Detection with ML.NET

Posted on by

This post covers change point detection, which is better suited for identifying persistent shifts in behaviour rather than momentary spikes. Specifically, comparing the two approaches available in ML.NET: Independent and identically distributed (IID) and Singular Spectrum Analysis (SSA) Change Point Detection

Change point detection is useful because IoT devices don’t always fail dramatically. Often, there’s a gradual or sustained change:

  • Temperature sensor starts reading consistently higher
  • A vibration sensor baseline shifts due to wear
  • Power consumption increases due to cooling issues

The sample application implements IID detection using DetectIidChangePoint and SSA with DetectChangePointBySsa.

IID assumes that: Incoming data points are independent and drawn from the same distribution. When the statistical properties shift (mean, variance), a change point is flagged.

AlgorithmLimitationsBest for
IIDDoesn’t account for trends or seasonality
Sensitive to noise		Fast detection
Simple signals
Low computational overhead
SSAComputationally expensive
Requires tuning		Signals with trends or seasonality
Noisy real-world IoT data
More robust long-term monitoring

SSA assumes that: The observed time‑series can be decomposed into a combination of trend, periodic components(seasonality), and noise. A change point is flagged when these components no longer match the historical structure of the series

BehaviourIIDSSA
Initial sensitivityHighModerate
Long-term behaviourAlways sensitiveAdapts
Noise tolerancePoorGood
Drift detectionKeeps firingFires once, then stops

In the application’s program.cs roughly the first 90 lines discusses changes to configuration to tune the models

var changePointEngine = _changePointEngines.GetOrAdd(subscribedTopic, _ =>
   new Lazy<TimeSeriesPredictionEngine<Model.TimeSeriesData, Model.ChangePointPrediction>>(() =>
   {
      try
      {
         switch (subscribedTopicSettings.DetectionMode)
         {
            case Model.DetectionMode.IID:
               var empty = _MLContext.Data.LoadFromEnumerable(new List<Model.TimeSeriesData>());

               IidChangePointEstimator iidPipe = _MLContext.Transforms.DetectIidChangePoint(
                              outputColumnName: nameof(Model.ChangePointPrediction.Prediction),
                              inputColumnName: nameof(Model.TimeSeriesData.Value),
                              confidence: subscribedTopicSettings.IIDSettings.Confidence,
                              changeHistoryLength: subscribedTopicSettings.IIDSettings.ChangeHistoryLength);

               var iidModel = iidPipe.Fit(empty);
               var iidEngine = iidModel.CreateTimeSeriesEngine<Model.TimeSeriesData, Model.ChangePointPrediction>(_MLContext);

               Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss:fff} Initialized IID change point engine for '{subscribedTopic}' conf:{subscribedTopicSettings.IIDSettings.Confidence}, changeHistory:{subscribedTopicSettings.IIDSettings.ChangeHistoryLength}, AnomalySide:{subscribedTopicSettings.IIDSettings.AnomalySide}");
               return iidEngine;

            case Model.DetectionMode.SSA:
               SsaChangePointEstimator ssaPipe = _MLContext.Transforms.DetectChangePointBySsa(
                                 outputColumnName: nameof(Model.ChangePointPrediction.Prediction),
                                 inputColumnName: nameof(Model.TimeSeriesData.Value),
                                 confidence: subscribedTopicSettings.SSASettings.Confidence,
                                 changeHistoryLength: subscribedTopicSettings.SSASettings.ChangeHistoryLength,
                                 trainingWindowSize: subscribedTopicSettings.SSASettings.TrainingWindowSize,
                                 seasonalityWindowSize: subscribedTopicSettings.SSASettings.SeasonalityWindowSize);

               var dataView = _MLContext.Data.LoadFromEnumerable(new List<Model.TimeSeriesData>());
               var ssaModel = ssaPipe.Fit(dataView);
               var ssaEngine = ssaModel.CreateTimeSeriesEngine<Model.TimeSeriesData, Model.ChangePointPrediction>(_MLContext);

               Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss:fff} Initialized SSA change point engine for '{subscribedTopic}' conf:{subscribedTopicSettings.SSASettings.Confidence}, changeHistory:{subscribedTopicSettings.SSASettings.ChangeHistoryLength}, trainingWindow:{subscribedTopicSettings.SSASettings.TrainingWindowSize}, seasonalityWindow:{subscribedTopicSettings.SSASettings.SeasonalityWindowSize}, AnomalySide:{subscribedTopicSettings.SSASettings.AnomalySide}");
               return ssaEngine;
            default:
               throw new NotSupportedException($"Detection mode {subscribedTopicSettings.DetectionMode} is not supported.");
         }

As part of a refactoring of configuration settings post my Real-time IoT Spike and Change Point Detection with ML.NET post the IID and SSA parameters have been split into separate classes

{
   "ApplicationSettings": {
      /*   */
      "SubscribedTopics": {
         "device/AshleyS1/distance": {
            "InputQualityOfService": 0,
            "OutputTopic": "alerts,ashley/changepoint",
            "OutputQualityOfService": 1,
            "ContentType": "application/json",
            "InputMessageTransformFile": "Transforms/InputSKU101991042.cs",
            "OutputMessageTransformFile": "Transforms/ChangePointOutput.cs",

            "DetectionMode": "IID", // IID or SSA
            "IIDSettings": {
               "Confidence": 95.0,
               "ChangeHistoryLength": 20,
               "AnomalySide": "TwoSided" //Positive, Negative,TwoSided
            },
            "SSASettings": {
               "Confidence": 95.0,
               "ChangeHistoryLength": 20,
               "SeasonalityWindowSize": 1,
               "TrainingWindowSize": 800,
               "AnomalySide": "TwoSided" //Positive, Negative,TwoSided
            }
         },
         "device/AshleyS72/distance": {
            "InputQualityOfService": 0,
            "OutputTopic": "alerts,ashley/changepoint",
            "OutputQualityOfService": 1,
            "ContentType": "application/json",
            "InputMessageTransformFile": "Transforms/InputSKU101991042.cs",
            "OutputMessageTransformFile": "Transforms/ChangePointOutput.cs",

            "DetectionMode": "IID",
            "IIDSettings": {
               "Confidence": 55.0,
               "ChangeHistoryLength": 10,
               "AnomalySide": "Negative" //Positive, Negative,TwoSided
            },
            "SSASettings": {
               "Confidence": 95.0,
               "ChangeHistoryLength": 20,
               "SeasonalityWindowSize": 2,
               "TrainingWindowSize": 300,
               "AnomalySide": "TwoSided" //Positive, Negative,TwoSided
            }
         }
      }
   }
}

In ML.NET time-series detectors (SSA and IID), the model doesn’t start detecting immediately. They first need to observe enough data to establish what “normal” looks like.

ParameterDescriptionTypical IoT Values
changeHistoryLengthNumber of points to evaluate stability30-100
trainingWindowSizeHistorical learning window (SSA)100-500
seasonalWindowSizeHistorical learning window (SSA)depends on cycle
confidenceSensitivity threshold90-99
AnomalySideStatistically unusual signalPositive, Negative, TwoSided

SSA requires a training window: trainingWindowSize: 200, this means the model needs roughly 200 samples before it is stable and before that predictions are unreliable and should be ignored. IID uses changeHistoryLength: 50 this means roughly 50 points are used to estimate the distribution and this is effectively the minimum history needed.

Warmup Matters: In the early stages the model is very sensitive as it has little context, and the baseline is weak which results in lots of detections and false positives (the detection below was after trainingWindowSize values). In the middle stage i.e. after enough samples the noise characteristics have been learned and patterns recognised. In the late stage after the model has been running for a long time it is very stable and less sensitive to change which can be an issue.

To cope with this (in a production system), I would ignore early predictions, for IID: changeHistoryLength and for SSA: trainingWindowSize + changeHistoryLength. To improve startup time preloading and SSA model with historical data or a synthetic base line is worth considering.

If an SSA model has been running for a longtime, or after a major change, reset the model to restore sensitivity. If sensitivity reduces over time it maybe worth considering “Dual Detectors: SSA for long-term structure, and IID for short-term sensitivity.

//---------------------------------------------------------------------------------
// Copyright (c) May 2026, devMobile Software
//
using System.Text.Json;
using System.Text.Json.Serialization; // Do not remove this using directive as it is required for the JsonIgnoreCondition

using devMobile.IoT.MqttTransformers;

public class ChangePointOutputTransformer : IChangePointOutputMessageTransformer
{
   private static readonly JsonSerializerOptions _serializerOptions = new() { DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull };

   public byte[] Transform(string sourceTopic, double value, double rawScore, double pValue, double martingale)
   {
      var obj = new
      {
         DetectionType = "ChangePoint",
         Value = value,
         SourceTopic = sourceTopic,
         RawScore = rawScore,
         PValue = pValue,
         Martingale = martingale,
      };

      string payload = JsonSerializer.Serialize(obj, _serializerOptions);

      return System.Text.Encoding.UTF8.GetBytes(payload);
   }
}

In the most recent version of the output transform ChangePoint = true was changed to DetectionType = “ChangePoint” to make it easier to identify change point messages for processing.

NOTE: SSA will not reliably detect slow, continuous movement – even if the total change is large

I was “tripped up” by this, it is not a bug, it’s an algorithm design choice to optimise for detecting regime changes, not motion. SSA is excellent at detecting when a system changes, but once it has adapted, it will treat that new state as completely normal.”

Mistral AI ASP Net CORE MinimalAPI Experiment

Posted on by

Over the last couple of months, I’ve been experimenting with a range of AI coding tools starting with GitHub Copilot, then Anthropic Claude, and more recently, Mistral (I was looking for an on-prem solution). Mistral is a French company so is covered by the General Data Protect Regulation(GDPR) rules of the European Union(EU) which are much better than the regulations other providers have to comply with.

Like many .NET developers, I started with Copilot as a natural extension of my workflow, expecting it to streamline repetitive tasks and accelerate development. When I started using Building Edge AI with Github Copilot- Security Camera HTTP(Jan 2025) the experience wasn’t great. Especially when I was using it for the “niche” areas I work-in it was pretty hopeless (sometimes even referred me to my own blog posts).

After a while I started trialing the other tools in my workflow and though they were better, sometimes F2-Replace or intellisense were faster and used a lot less tokens. I would also get the tools to review the code of the others, and I especially liked the Claude “Irony stack” when using it review Co-Pilot generated code.

While the other tools certainly helped (especially after adding custom skills files), I often found myself spending as much time going “down rabbit holes”(not the tool’s problem, though I hopefully learnt some useful stuff) and correcting or restructuring or debugging generated code that I could have written faster from scratch.

That’s what made my “out of box” experience with Mistral stand out. With a relatively simple prompt, it produced code that was not only concise but surprisingly accurate with just a single compile time error and no warnings on the first pass.

NOTE: This was using the webby interface, but I now have a paid for subscription.

The instructions which included .NET 8 (bit retro) and “dotnet add package”(pretty good) meant the code compiled on second attempt. The issue was a syntax error initialising OpenTelemetry which was quickly fixed, somewhat ironically with GitHub Copilot.

.ConfigureResource(resourceBuilder) rather than .ConfigureResource(rb => rb = resourceBuilder) 

//dotnet add package OpenTelemetry 
//dotnet add package OpenTelemetry.Extensions.Hosting
//dotnet add package OpenTelemetry.Instrumentation.AspNetCore
//dotnet add package OpenTelemetry.Instrumentation.Http
//dotnet add package OpenTelemetry.Exporter.Console
//dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
//
//using OpenTelemetry;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using System.Diagnostics;

var builder = WebApplication.CreateBuilder(args);

// Configure OpenTelemetry with a resource (service name)
var resourceBuilder = ResourceBuilder.CreateDefault()
    .AddService(serviceName: builder.Environment.ApplicationName);

// Add OpenTelemetry Tracing
builder.Services.AddOpenTelemetry()
    //.ConfigureResource(resourceBuilder) /**** This was the only compile time issue
    .ConfigureResource(rb => rb = resourceBuilder)
    .WithTracing(tracerProviderBuilder =>
    {
       tracerProviderBuilder
           .AddSource("MinimalApiSample")
           .AddAspNetCoreInstrumentation(options =>
           {
              options.RecordException = true;
           })
           .AddHttpClientInstrumentation()
           .AddConsoleExporter(); // For demo: export to console
           //.AddOtlpExporter(); // Uncomment to export to OpenTelemetry Collector

    })
    .WithMetrics(metricsProviderBuilder =>
    {
       metricsProviderBuilder
           .AddAspNetCoreInstrumentation()
           .AddHttpClientInstrumentation()
           .AddConsoleExporter(); // For demo: export to console
           //.AddOtlpExporter(); // Uncomment to export to OpenTelemetry Collector
    });

var app = builder.Build();

// Example of a custom activity for tracing
var activitySource = new ActivitySource("MinimalApiSample");

app.MapGet("/", () =>
{
   using var activity = activitySource.StartActivity("RootEndpoint");
   activity?.SetTag("custom.tag", "Hello, OpenTelemetry!");
   return Results.Ok("Hello, OpenTelemetry!");
});

app.MapGet("/metrics", () =>
{
   // This endpoint is just for demo; metrics are exported automatically
   return Results.Ok("Metrics are being collected in the background.");
});

app.Run();

//dotnet add package OpenTelemetry
//dotnet add package OpenTelemetry.Extensions.Hosting
//dotnet add package OpenTelemetry.Instrumentation.AspNetCore
//dotnet add package OpenTelemetry.Instrumentation.Http
//dotnet add package Azure.Monitor.OpenTelemetry.Exporter
//
using Azure.Monitor.OpenTelemetry.Exporter; 
//using OpenTelemetry; 
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using System.Diagnostics;

var builder = WebApplication.CreateBuilder(args);

// Configure OpenTelemetry with a resource (service name)
var resourceBuilder = ResourceBuilder.CreateDefault()
    .AddService(serviceName: builder.Environment.ApplicationName)
    .AddTelemetrySdk();

// Add OpenTelemetry Tracing and Metrics for Azure Application Insights
builder.Services.AddOpenTelemetry()
    //.ConfigureResource(resourceBuilder)
    .ConfigureResource(rb => rb = resourceBuilder) //*****
    .WithTracing(tracerProviderBuilder =>
    {
       tracerProviderBuilder
           .AddSource("MinimalApiSample")
           .AddAspNetCoreInstrumentation(options =>
           {
              options.RecordException = true;
           })
           .AddHttpClientInstrumentation()
           .AddAzureMonitorTraceExporter(options =>
           {
              options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
           });
    })
    .WithMetrics(metricsProviderBuilder =>
    {
       metricsProviderBuilder
           .AddAspNetCoreInstrumentation()
           .AddHttpClientInstrumentation()
           .AddAzureMonitorMetricExporter(options =>
           {
              options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
           });
    });

var app = builder.Build();

// Example of a custom activity for tracing
var activitySource = new ActivitySource("MinimalApiSample");

app.MapGet("/", () =>
{
   using var activity = activitySource.StartActivity("RootEndpoint");
   activity?.SetTag("custom.tag", "Hello, Azure Application Insights!");
   return Results.Ok("Hello, Azure Application Insights!");
});

app.MapGet("/metrics", () =>
{
   return Results.Ok("Metrics and traces are being sent to Azure Application Insights.");
});

app.Run();

Using Application Insights metrics the Kestral.active_connections graphs to shows some of the additional telemetry emitted by the application.

//dotnet add package OpenTelemetry
//dotnet add package OpenTelemetry.Extensions.Hosting
//dotnet add package OpenTelemetry.Instrumentation.AspNetCore
//dotnet add package OpenTelemetry.Instrumentation.Http
//dotnet add package Azure.Monitor.OpenTelemetry.Exporter
//
using Azure.Monitor.OpenTelemetry.Exporter;
//using OpenTelemetry;  
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using System.Diagnostics;
using System.Diagnostics.Metrics;

var builder = WebApplication.CreateBuilder(args);

// Configure OpenTelemetry with a resource (service name)
var resourceBuilder = ResourceBuilder.CreateDefault()
    .AddService(serviceName: builder.Environment.ApplicationName)
    .AddTelemetrySdk();

// Create a meter for custom metrics
var meter = new Meter("MinimalApiSample.Metrics");
var metricsCounter = meter.CreateCounter<int>("MetricsEndpointAccessCount");

// Add OpenTelemetry Tracing and Metrics for Azure Application Insights
builder.Services.AddOpenTelemetry()
    //.ConfigureResource(resourceBuilder) /**** This is the only compile time issue
    .ConfigureResource(rb=>rb =  resourceBuilder)
    .WithTracing(tracerProviderBuilder =>
    {
       tracerProviderBuilder
           .AddSource("MinimalApiSample")
           .AddAspNetCoreInstrumentation(options =>
           {
              options.RecordException = true;
           })
           .AddHttpClientInstrumentation()
           .AddAzureMonitorTraceExporter(options =>
           {
              options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
           });
    })
    .WithMetrics(metricsProviderBuilder =>
    {
       metricsProviderBuilder
           .AddAspNetCoreInstrumentation()
           .AddHttpClientInstrumentation()
           .AddMeter("MinimalApiSample.Metrics") // Add your custom meter
           .AddAzureMonitorMetricExporter(options =>
           {
              options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
           });
    });

var app = builder.Build();

// Example of a custom activity for tracing
var activitySource = new ActivitySource("MinimalApiSample");

app.MapGet("/", () =>
{
   using var activity = activitySource.StartActivity("RootEndpoint");
   activity?.SetTag("custom.tag", "Hello, Azure Application Insights!");
   return Results.Ok("Hello, Azure Application Insights!");
});

app.MapGet("/metrics", () =>
{
   // Increment custom metric on each access
   metricsCounter.Add(1);
   return Results.Ok("Metrics and traces are being sent to Azure Application Insights.");
});

app.Run();

I by pleasantly surprised by suggestion of a counter for each endpoint which was my original intent.

Couldn’t think of a better name “scirtem” is “metrics” backwards. The way Meter and CreateCount are global would not be a good idea in a more complex system but this is fine for a hacky PoC.

//dotnet add package OpenTelemetry
//dotnet add package OpenTelemetry.Extensions.Hosting
//dotnet add package OpenTelemetry.Instrumentation.AspNetCore
//dotnet add package OpenTelemetry.Instrumentation.Http
//dotnet add package Azure.Monitor.OpenTelemetry.Exporter
//
using Azure.Monitor.OpenTelemetry.Exporter;
//using OpenTelemetry;  //***** Unnecessary with OpenTelemetry.Extensions.Hosting
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using System.Diagnostics;
using System.Diagnostics.Metrics;

var builder = WebApplication.CreateBuilder(args);

// Configure OpenTelemetry with a resource (service name)
var resourceBuilder = ResourceBuilder.CreateDefault()
    .AddService(serviceName: builder.Environment.ApplicationName)
    .AddTelemetrySdk();

// Create a meter for custom metrics
var meter = new Meter("MinimalApiSample.Metrics");
var metricsCounter = meter.CreateCounter<int>("MetricsEndpointAccessCount");
var scirtemCounter = meter.CreateCounter<int>("ScirtemEndpointAccessCount");

// Add OpenTelemetry Tracing and Metrics for Azure Application Insights
builder.Services.AddOpenTelemetry()
    //.ConfigureResource(resourceBuilder) /**** This is the only compile time issue
    .ConfigureResource(rb=>rb =  resourceBuilder)
    .WithTracing(tracerProviderBuilder =>
    {
       tracerProviderBuilder
           .AddSource("MinimalApiSample")
           .AddAspNetCoreInstrumentation(options =>
           {
              options.RecordException = true;
           })
           .AddHttpClientInstrumentation()
           .AddAzureMonitorTraceExporter(options =>
           {
              options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
           });
    })
    .WithMetrics(metricsProviderBuilder =>
    {
       metricsProviderBuilder
           .AddAspNetCoreInstrumentation()
           .AddHttpClientInstrumentation()
           .AddMeter("MinimalApiSample.Metrics") // Add your custom meter
           .AddAzureMonitorMetricExporter(options =>
           {
              options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
           });
    });

var app = builder.Build();

// Example of a custom activity for tracing
var activitySource = new ActivitySource("MinimalApiSample");

app.MapGet("/", () =>
{
   using var activity = activitySource.StartActivity("RootEndpoint");
   activity?.SetTag("custom.tag", "Hello, Azure Application Insights!");
   return Results.Ok("Hello, Azure Application Insights!");
});

app.MapGet("/metrics", () =>
{
   // Increment custom metric on each access
   metricsCounter.Add(1);
   return Results.Ok("Metrics and traces are being sent to Azure Application Insights.");
});

app.MapGet("/scirtem", () =>
{
   scirtemCounter.Add(1);
   return Results.Ok("Scirtem endpoint accessed.");
});

app.Run();

Using Application Insights metrics the MetricsEndPointAccesCount, and ScirtemEndPointAccesCount, plots to show the OLTP telemetry emitted by the application.

Mistral generated the code for the endpoint latency histogram without any prompting.

//dotnet add package OpenTelemetry
//dotnet add package OpenTelemetry.Extensions.Hosting
//dotnet add package OpenTelemetry.Instrumentation.AspNetCore
//dotnet add package OpenTelemetry.Instrumentation.Http
//dotnet add package Azure.Monitor.OpenTelemetry.Exporter
//
using Azure.Monitor.OpenTelemetry.Exporter;
//using OpenTelemetry;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using System.Diagnostics;
using System.Diagnostics.Metrics;

var builder = WebApplication.CreateBuilder(args);

// Configure OpenTelemetry with a resource (service name)
var resourceBuilder = ResourceBuilder.CreateDefault()
    .AddService(serviceName: builder.Environment.ApplicationName)
    .AddTelemetrySdk();

// Create a meter for custom metrics
var meter = new Meter("MinimalApiSample.Metrics");
var metricsCounter = meter.CreateCounter<int>("MetricsEndpointAccessCount");
var scirtemCounter = meter.CreateCounter<int>("ScirtemEndpointAccessCount");
var histogram = meter.CreateHistogram<double>("HistogramEndpointLatencyMs");

// Add OpenTelemetry Tracing and Metrics for Azure Application Insights
builder.Services.AddOpenTelemetry()
    //.ConfigureResource(resourceBuilder) /**** This is the only compile time issue
    .ConfigureResource(rb=>rb =  resourceBuilder)
    .WithTracing(tracerProviderBuilder =>
    {
       tracerProviderBuilder
           .AddSource("MinimalApiSample")
           .AddAspNetCoreInstrumentation(options =>
           {
              options.RecordException = true;
           })
           .AddHttpClientInstrumentation()
           .AddAzureMonitorTraceExporter(options =>
           {
              options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
           });
    })
    .WithMetrics(metricsProviderBuilder =>
    {
       metricsProviderBuilder
           .AddAspNetCoreInstrumentation()
           .AddHttpClientInstrumentation()
           .AddMeter("MinimalApiSample.Metrics") // Register your custom meter
           .AddAzureMonitorMetricExporter(options =>
           {
              options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
           });
    });

var app = builder.Build();

// Example of a custom activity for tracing
var activitySource = new ActivitySource("MinimalApiSample");

app.MapGet("/", () =>
{
   using var activity = activitySource.StartActivity("RootEndpoint");
   activity?.SetTag("custom.tag", "Hello, Azure Application Insights!");
   return Results.Ok("Hello, Azure Application Insights!");
});

app.MapGet("/metrics", () =>
{
   metricsCounter.Add(1);
   return Results.Ok("Metrics endpoint accessed.");
});

app.MapGet("/scirtem", () =>
{
   scirtemCounter.Add(1);
   return Results.Ok("Scirtem endpoint accessed.");
});

app.MapGet("/histogram", async () =>
{
   // Simulate some work
   var startTime = Stopwatch.GetTimestamp();
   await Task.Delay(Random.Shared.Next(50, 200)); // Random delay between 50-200ms
   var endTime = Stopwatch.GetTimestamp();

   // Calculate latency in milliseconds
   var latencyMs = (endTime - startTime) * 1000.0 / Stopwatch.Frequency;
   histogram.Record(latencyMs);

   return Results.Ok($"Histogram endpoint accessed. Latency: {latencyMs:F2}ms");
});

app.Run();

Using Application Insights metrics the OpenTelemetry.HistogramEndpointLatencyMs plot to show the OLTP telemetry emitted by the application.

Even with my relatively trivial OTLP learning applications, Mistral consistently produced clean and usable code with my simple prompts (maybe, I have got better and prompting). The generated code was straightforward, required only minor fixes, and avoided much of the over-complexity I’d seen in earlier experiments with other tools (looking at you mid/late 2025 Copilot). For my simple OTLP observability learning scenarios, that translated into faster iteration and less time spent refactoring and debugging generated code.

Real-time IoT Spike and Change Point Detection with ML.NET

Posted on by

Internet of Things(IoT) sensors generate streams of continuous numeric data. I wanted to detect spikes and change points without building full a Machine Learning (ML) pipeline per device. My approach is to subscribe to Message Queue Telemetry Transport (MQTT) topics using HiveMQ .NET Client Library, extract each message’s numeric value using CS-Script, process it with an ML.NET time-series prediction engine then finally publish an alert to a MQTT Topic when a spike is detected.

This post details the plumbing for detecting change points and spikes;

The next post will cover spike detection with Independent and identically distributed (IID) using DetectIidSpike, and Singular Spectrum Analysis (SSA) with DetectSpikeBySsa.

The final post will cover change point detection with Independent and identically distributed (IID) using DetectIidChangePoint. and Singular Spectrum Analysis (SSA) with DetectChangePointBySsa .

The configuration is stored in appsettings.json and when running on a desktop sensitive information AddUserSecrets() keeps credentials out of source control. The HiveMQ client configured with the builder pattern: The WithClientId, WithBroker, WithPort, WithCleanStart, WithAutomaticReconnect and WithUseTls are loaded from the appsettings.json. To stop the compiler whining about “deprecated methods” there are compile-time #if guards for username/password and certificate blocks.

// Load configuration
var configuration = new ConfigurationBuilder()
         .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
         .AddUserSecrets<Program>()
         .Build();

_applicationSettings = configuration.GetSection("ApplicationSettings").Get<Model.ApplicationSettings>() ?? throw new Exception("ApplicationSettings not configured");

// HiveMQ client options
var optionsBuilder = new HiveMQClientOptionsBuilder()
         .WithClientId(_applicationSettings.ClientId)
         .WithBroker(_applicationSettings.Host)
         .WithPort(_applicationSettings.Port)
#if HIVEMQ_USERNAME_AND_PASSWORD_SUPPORT
         .WithUserName(_applicationSettings.UserName)
         .WithPassword(_applicationSettings.Password)
#endif
         .WithCleanStart(_applicationSettings.CleanStart)
         .WithAutomaticReconnect(_applicationSettings.AutomaticReconnect)
         .WithUseTls(_applicationSettings.UseTls);

#if HIVEMQ_CERTIFICATE_SUPPORT
if (!string.IsNullOrWhiteSpace(_applicationSettings.ClientCertificateFileName))
{
      optionsBuilder.WithClientCertificate(_applicationSettings.ClientCertificateFileName,_applicationSettings.ClientCertificatePassword);
   }
#endif

#if HIVEMQ_USERNAME_AND_PASSWORD_SUPPORT
if (!string.IsNullOrWhiteSpace(_applicationSettings.UserName))
{
   optionsBuilder = optionsBuilder.WithPassword(_applicationSettings.Password);
}
#endif

The ApplicationSettings.SubscribedTopics are loaded into a dictionary with the MQTT topic as the key. Each subscription has per-topic configuration (detection mode, confidence, window sizes, QoS, output topic).

The HIVEMQ_CERTIFICATE_SUPPORT and HIVEMQ_USERNAME_AND_PASSWORD_SUPPORT exclude calls to WithUserName, WithPassword and WithClientCertificate(fileName, password). This is required (from version 0.39 of the HiveMQtt .NETClient) even though SecureString is considered obsolete in .NET for cross-platform work.

Converting a plain string from appsettings.json into a SecureString is a PITA as you must iterate characters and AppendChar one by one, then call MakeReadOnly(). SecureString limits the window during which the plaintext lives in managed heap memory,
reducing exposure to memory dumps. I don’t think that trade-off is worth the ceremony is debatable for most IoT scenarios.

Each subscribed topic has two scripts (InputMessageTransformFile and OutputMessageTransformFile) which CSScript.Evaluator.LoadFile(path) compiles, caches and instantiates a C# script at runtime

The IInputMessageTransformer, ISpikeOutputMessageTransformer follow the same pattern for the inbound and outbound payloads. The interfaces decouple the spike detection core from message format. A new or updated .cs script file to handle a new sensor payload can be deployed without touching the host application.

Exception handling is non-optional: script compilation errors surface as exceptions. The try/catch around the foreach that loads transformers is the right place to fail fast at startup rather than silently skipping a topic.

Console.CancelKeyPress sets e.Cancel = true (which suppresses the default termination) and calls cts.Cancel(). Await Task.Delay(Timeout.Infinite, cts.Token), the program does nothing in Main while the event-driven OnMessageReceived handler does all the work. When Ctrl+C fires, Task.Delay throws TaskCanceledException, which is caught in the outer try/catch block and prints a clean shutdown message. Everything else falls through to finally, which disconnects the HiveMQ client.

// one engine per topic; keeps rolling state for IID detector
private static readonly ConcurrentDictionary<string, Lazy<TimeSeriesPredictionEngine<Model.TimeSeriesData, Model.SpikePrediction>>> _spikeEngines = new();

// lock per topic because TimeSeriesPredictionEngine is not thread-safe
private static readonly ConcurrentDictionary<string, object> _engineLocks = new();

...

var spikeEngine = _spikeEngines.GetOrAdd(subscribedTopic, _ =>
   new Lazy<TimeSeriesPredictionEngine<Model.TimeSeriesData, Model.SpikePrediction>>(() =>
or...
var changePointEngine = _changePointEngines.GetOrAdd(subscribedTopic, _ =>
   new Lazy<TimeSeriesPredictionEngine<Model.TimeSeriesData, Model.ChangePointPrediction>>(() =>
{

...

}, LazyThreadSafetyMode.PublicationOnly)).Value;

The _spikeEngines/_changePointEngines is a ConcurrentDictionary keyed by topic string, GetOrAdd with a Lazy<> factory ensures each engine is initialised exactly once, even under concurrent message arrival. The LazyThreadSafetyMode.PublicationOnly means multiple threads may race to construct the value but only one result survives.

Each engine carries rolling internal state (the sliding window of past values), so isolation per topic is essential. The “mixing” of values from different sensors(topics) into one engine would produce nonsense predictions. The DetectionMode enumeration (IID / SSA) is checked inside the Lazy factory, branching to the appropriate ML.NET estimator. The engine type is the same either way (TimeSeriesPredictionEngine) only the output transform and pipeline construction differs.

Two separate try/catch blocks wrap the two transforms in each program, so failures (runtime errors like, bad JSON, missing field, type mismatch etc.) don’t crash the handler or the engine.

float value;
try
{
   value = subscribedTopicSettings.InputMessageTransformer.Transform(subscribedTopic, e.PublishMessage.Payload);
}
catch (Exception ex)
{
   Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss:fff} Input transform failed: {ex.Message}");
   return;
}

foreach (string topic in topics)
{
   Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss:fff} Publishing to:{topic}");

   var message = new MQTT5PublishMessage(topic, subscribedTopicSettings.OutputQualityOfService)
   {
      ContentType = subscribedTopicSettings.ContentType,
      Payload = payload
   };

   try
   {
      var resultPublish = await client.PublishAsync(message);

      Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss:fff} Published:{resultPublish.QoS1ReasonCode} {resultPublish.QoS2ReasonCode}");
   }
   catch (Exception ex)
   {
      Console.WriteLine($"{DateTime.UtcNow:yy-MM-dd HH:mm:ss:fff} client.PublishAsync failed: {ex.Message}");
      return;
   }
}

The async void OnMessageReceived wrapper catches any unhandled exception from OnMessageReceivedCoreAsync. This is necessary because the async void “swallows” exceptions that would otherwise terminate the process silently. This approach is required because a bad/broken payload from one device should never take down detection for all other topics.

Christchurch Azure User Group Session April 2026

Posted on by

Faster, Cheaper, Scalable: Architecting High-Performance Azure Apps with Caching

Details

“There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.” — Leon Bambrick

Join us as Microsoft MVP Bryn Lewis shows us how caching is the ultimate “cheat code” for cloud architecture. When implemented correctly, it’s the fastest way to slash your Azure consumption costs, reduce database contention, and keep your application responsive under massive load. But move beyond simple lookups – your deployment model and caching strategy can make or break your app’s reliability. In this session, we’ll move from the browser edge to the distributed core:

  • Optimizing the Edge: Leverage RFC-standard HTTP semantics to offload traffic to CDNs and browsers, cutting ingress/egress costs before requests even reach your App Service.
  • Saving Compute: See how ASP.NET Core Output Caching rescues your CPU from redundant work, allowing you to scale out less frequently and save on your monthly Azure bill.
  • Modern Object Strategies: A deep dive into HybridCache and FusionCache. We’ll compare L1/L2 strategies and master “the dark arts” of stampede protection and cache invalidation to ensure high availability.
  • The Power of Azure Cache for Redis: We’ll close by configuring Redis as a distributed L2 cache, ensuring your cloud applications stay fast, synchronized, and resilient across multiple instances.

The code I used to double check my assumptions is available on GitHub. This repo demonstrates various .NET caching strategies (FusionCache, HybridCache, OutputCache, Redis, etc.) against a real Azure SQL Server backend.

Dapper Extensions

All the demo projects use my DapperExtensions project, so every cache benchmark hits the database through the same resilient layer, meaning the retry logic “should never” skew the results.

DIYCache – Rolling your own cache in 80 lines

The cache is a ConcurrentDictionary<string,CacheItem<T>>> registered as a singleton. The GET endpoint uses the cache-aside pattern, return if valid and not expired, otherwise hit the database, and store the result with a configurable TTL, then return. A companion DELETE endpoint evicts a specific entry with a single TryRemove call. The cache has no background eviction, or stampede protection, and the size is “unbounded”

Fusion Cache – Scale with configuration

FusionCache is a read through cache with Fast L1/Shared L2 support which hides the checking, fetching, and storing in three separate steps, with a factory lambda to manage the process. Cache invalidation uses tags, each entry is stamped at write time, and a single RemoveByTagAsync call evicts every matching entry. In the sample project Stack Exchange Redis is opt-in via configuration. Add the required connection string and FusionCache becomes a two-tier cache: fast in-memory L1 backed by distributed Redis L2. Add a backplane connection string and invalidation signals propagate across all running instances. The same code works as a single-process cache in development and a fully distributed one in production with no code changes. Realistically it was what I was hoping HybridCache would be

HTTP Head – RFC 9111 IETF “HTTP Caching”.

The HTTPHead project shows how HTTP’s HEAD method and ETags can eliminate unnecessary data transfers. When a client fetches a neighborhood record via GET, it receives an ETag derived from the Azure SQL Server rowversion (replaces the TimeStamp which has been deprecated) column. On subsequent checks, it sends that ETag to the HEAD endpoint, which queries only the version column and returns 304 Not Modified or 200 OK no payload needed. The PUT endpoint uses optimistic concurrency, rejecting updates where the ETag no longer matches. This ensures clients only download data that has actually changed.

Hybrid Cache – If only

HybridCache is a two-tier cache that sits in front of both an in-process L1 cache and an optional Redis L2 cache behind a single GetOrCreateAsync call. In the sample code NeighborHood lookups are cached with a 5-minute in-memory expiry and a 30-minute Stack Exchange Redis expiry, so repeated requests within the same process never leave the machine, while distributed deployments still share a warm cache across instances.

Hybrid Cache Serialization – When less sent to the L2 Cache is more

The HybridCacheSerialization project extends the HybridCache sample by swapping the default JSON serializer for others like Neuecc MessagePack. HybridCache exposes an IHybridCacheSerializer interface, so developers can plug in different serialisers. In the sample the Data Transfer Object(DTO) is decorated with [MessagePackObject] and [Key(n)] attributes to control the binary layout (MessagePack message format is supported by many languages). The payoff is compact, fast binary payloads stored in Stack Exchange Redis instead of verbose JSON. This is worthwhile when cached objects are large, retrieved frequently, or bandwidth between app and cache is a latency/jitter/cost concern.

Object Cache – Barely sufficient

The ObjectCache project is the simplest (non-DIY) option using just IMemoryCache. Neighborhood lookups are wrapped in GetOrCreateAsync: a hit returns the cached object instantly, a miss queries Azure SQL Server and caches the result for 5 minutes. In this example a database miss isn’t just returned as NotFound and forgotten, this is cached too, for 1 minute, so a flood of requests for a non-existent record won’t hammer the database. A DELETE endpoint lets callers evict a specific entry on demand.

Output Cache – Avoiding regeneration, but don’t cross the streams.

The OutputCache project demonstrates ASP.NET Core’s OutputCaching middleware, a response-level cache that stores the fully serialized HTTP responses rather than the underlying objects. Output caching short-circuits the entire endpoint and serves the cached bytes directly. The project has named policies (“short”, “medium”, “neighborhood”) defined at startup and applied to endpoints with .CacheOutput(), inline policies defined inline as a lambda, Stack Exchange Redis can be dropped in
as the backing store with no code changes

MIDDLEWARE ORDER MATTERS- Place AFTER authentication/authorization so user identity and policies respected

Redis Cache – Old school and amazingly Fast

The RedisCache project goes bare-metal, using the Stack Exchange Redis IConnectionMultiplexer directly rather than any .NET caching abstraction. The cache-aside pattern used, check Redis first, then fall back to the database on a miss, then write the result back with a 30-second TTL. This sample uses source-generated JSON serializationvia JsonSerializerContext: serialization and deserialization use pre-compiled code paths rather than runtime reflection, which keeps allocation low and throughput high on the hot path. This also enables Ahead of Time(AoT) compilation support.

ResponseCache – RFC 9110 IETF “HTTP Semantics”

The ResponseCache project covers ASP.NET Core’s older ResponseCaching middleware, which caches responses based on standard HTTP Cache-Control headers rather than any framework-specific API. The endpoint sets Cache-Control: public, max-age=90 directly on the response headers and the middleware handles the rest. ResponseCache has largely been replaced by Output Cache though it matters when managing the caching behaviour of downstream proxies and Content Delivery Networks(CDNs), because the Cache-Control headers it emits are understood by the full HTTP stack.

Response Compression – When less sent to the client is more

The ResponseCompression middleware is server-side complement to caching that reduces payload size rather than request database traffic. The sample supports Gzip (faster,universally supported) and Brotli (better compression ratio, higher CPU cost), with an optionalflag to tune the trade-off between speed and size.

The application/json content-type isn’t compressed by default so it must be added explicitly to the Multipurpose Internet Mail Extension(MIME) type list; EnableForHttps must be opted into deliberately since compressing encrypted responses can expose reflected secrets (the CRIME/BREACH attacks); and Azure App Service containers apply their own platform-level gzip, so enabling this middleware there risks double-compression. Clients must send Accept-Encoding: gzip for compression as it’s not automatic.

The full source is available in the CHCAzureUGC202604 repository alongside the caching demos it supports

Arduino RS485 Light Intensity Sensor

Posted on by

As part of this series of samples comparing Arduino to nanoFramework to .NET IoT Device “Proof of Concept (PoC) applications, several posts use a SenseCap Industrial Light Intensity Sensor (SKU314990739 or SKU 314990740)

I cut one of the cables of a spare Industrial IP68 Modbus RS485 1-to-4 Splitter/Hub to connect the sensor to the RS485 breakout board. The sensor has an operating voltage of 3.6-30V but it is connected to the 12V supply pin. Initially, I had the sensor connected to the 5V output of the RS485 Breakout Board for Seeed Studio XIAO (SKU 113991354) so it didn’t work. 

HardwareSerial RS485Serial(1);
ModbusMaster node;

// -----------------------------
// RS485 Pin Assignments (Corrected)
// -----------------------------
const int RS485_RX = 6; // UART1 RX
const int RS485_TX = 5; // UART1 TX
const int RS485_EN = D2;

// Sensor/Modbus parameters (from datasheet)
#define MODBUS_SLAVE_ID 0x0d
#define NUMBER_OF_REGISTERS_TO_READ 0x03
#define REG_LUX_HIGH 0x0000
#define REG_LUX_LOW 0x0001
#define REG_STATUS 0x0002

// Forward declarations for ModbusMaster callbacks
void preTransmission();
void postTransmission();


void setup() {
  Serial.begin(9600);
  delay(5000);

  Serial.println("ModbusMaster: Seeed SKU 314990740 starting");

  // Wait for the hardware serial to be ready
  while (!Serial)
    ;
  Serial.println("Serial done");

  pinMode(RS485_EN, OUTPUT);
  digitalWrite(RS485_EN, LOW);  // Start in RX mode

  // Datasheet: 9600 baud, 8N1
  RS485Serial.begin(9600, SERIAL_8N1, RS485_RX, RS485_TX);
  while (!RS485Serial)
    ;
  Serial.println("RS485 done");

  // Tie ModbusMaster to the UART we just configured
  node.begin(MODBUS_SLAVE_ID, RS485Serial);

   // Register callbacks for half-duplex direction control
  node.preTransmission(preTransmission);
  node.postTransmission(postTransmission);
}

// Toggle DE/RE around TX per ModbusMaster design
void preTransmission() {
  digitalWrite(RS485_EN, HIGH);  // enable driver (TX)
  delayMicroseconds(250);        // transceiver turn-around margin
}

void postTransmission() {
  delayMicroseconds(250);       // ensure last bit left the wire
  digitalWrite(RS485_EN, LOW);  // back to receive
}

void loop() {

  uint8_t result = node.readInputRegisters(REG_LUX_HIGH, NUMBER_OF_REGISTERS_TO_READ);

  if (result == node.ku8MBSuccess) {
    uint16_t luxHigh = node.getResponseBuffer(REG_LUX_HIGH) ;
    
    uint16_t luxLow = node.getResponseBuffer(REG_LUX_LOW);

    uint16_t status = node.getResponseBuffer(REG_STATUS);
    
    uint32_t lux = ((uint32_t)luxHigh << 16) | luxLow;

    Serial.printf("illuminance: %u lux Status:%02X \n", lux, status);
  }
  else
  {
    Serial.printf("Modbus error: %d\n", result);      
  }

  delay(60000);
}

I do wonder how “accurately” the sensor has to be mounted because it has a level indicator and spring-loaded bolts.

ONNX Tensor loading Initial Comparison

Posted on by

This is the second in a series of posts from my session at the Agent Camp – Christchurch about using Open Neural Network Exchange(ONNX) for processing Moving Picture Experts Group (MPEG) video and Pulse Code Modulation(PCM) audio streams.

These benchmarks use Ultralytics Yolo26 standard object detection model input image size of 640*640pixels.

var _tensor= new DenseTensor<float>(new[] { 1, 3, modelH, modelW });

The original nested loop: multi-dimensional [0,c,y,x] indexer, with divide by 255f. This is the baseline to measure all other implementations against.

[Benchmark(Baseline = true, Description = "Baseline: indexer + / 255f")]
public void Baseline()
{
   for (int y = 0; y < modelH; y++)
      for (int x = 0; x < modelW; x++)
      {
          var c = _letterboxed.GetPixel(x, y);

         _tensor[0, 0, y, x] = px.Red / 255f;
         _tensor[0, 1, y, x] = px.Green / 255f;
         _tensor[0, 2, y, x] = px.Blue / 255f;
      }
}

The implementation bypasses the multi-dimensional [0,c,y,x] indexer entirely with Span<> over the tensor’s backing buffer. Channel planes are at offsets 0, planeSize, and 2*planeSize. Then a single loop reads each pixel once; writes to all three planes interleaved. 

[Benchmark(Description = "Buffer span: flat index, interleaved")]
public void BufferSpan()
{
   SKColor[] pixels = _letterboxed.Pixels;
   const float scaler = 1 / 255f;
   int planeSize = _modelW* _modelW;
   Span<float> buf = _tensor.Buffer.Span;

   for (int i = 0; i < planeSize; i++)
   {
      SKColor px = pixels[i];
      buf[i] = px.Red * scaler;
      buf[planeSize + i] = px.Green * scaler;
      buf[2 * planeSize + i] = px.Blue * scaler;
   }
}

This implementation slices the flat buffer into three non-overlapping channel spans, it then runs three separate sequential loops, one for each colour. This Combines the benefits of span (no indexer overhead, JIT can also auto-vectorise) and with split loops which the JIT can eliminate per-element bounds checks after the slice.

   [Benchmark(Description = "Buffer span split: 3× sequential flat loops")]
   public void BufferSpanSplit()
   {
      SKColor[] pixels = _letterboxed.Pixels;
      const float scaler = 1 / 255f;
      int planeSize = _modelW* _modelH;
      Span<float> buf = _tensor.Buffer.Span;

      Span<float> rPlane = buf.Slice(0, planeSize);
      Span<float> gPlane = buf.Slice(planeSize, planeSize);
      Span<float> bPlane = buf.Slice(2 * planeSize, planeSize);

      for (int i = 0; i < planeSize; i++) rPlane[i] = pixels[i].Red * scaler;
      for (int i = 0; i < planeSize; i++) gPlane[i] = pixels[i].Green * scaler;
      for (int i = 0; i < planeSize; i++) bPlane[i] = pixels[i].Blue * scaler;
   }

The minimal difference in performance of the two fastest implementations of the benchmark suite running on my development box was a surprise. It will be interesting to see how the performance of the different implementations changes on my Seeedstudio EdgeBox RPi 200 which has a different instruction set (esp. ARM NEON Single Instruction, Multiple Data (SIMD) extensions) and memory caching model

These benchmarks should be treated as indicative not authoritative 

SkiaSharp and ImageSharp Initial Comparison

Posted on by

This is the first in a series of posts from my session at the Agent Camp – Christchurch about using Open Neural Network Exchange(ONNX) for processing Moving Picture Experts Group (MPEG) video and Pulse Code Modulation(PCM) audio streams.

For processing video streams one of the first steps is extracting individual Joint Photographic Experts Group(JPEG) images from MPEG Real-Time Streaming Protocol(RTSP) stream. The jpeg images then have to transformed into an ONNX DenseTensor<float> in the correct format for the Ultralytics Yolo26 model. These image processing posts will use Ultralytics Yolo26 standard Small object detection model which has an input image size of 640*640pixels.

I have used both the YoloSharp and YoloDotNet libraries (Thank you Niklas Swärd and dme-compunet I appreciate the amount of effort you have put in). Both these libraries have support for object detection, instance segmentation, oriented bounding boxes detection(OBB), classification and pose estimation. They both have support for different versions, video stream processing, plotting minimum bounding boxes, Non-Maximum Suppression(NMS) for earlier models like YOLOv8 or YOLO11. I just need object detection (none of the other model types, plotting minimum boxes etc.) to work as fast as possible on my Seeedstudio EdgeBox RPi 200.

First step, was to use Benchmark.Net compare the performance of Six Labors ImageSharp (used by YoloSharp) and SkiaSharp (used by YoloDotNet). Six Labors ImageSharp  is a high-performance, fully managed, 2D graphics API whereas SkiaSharp is a wrapper for Google’s Skia 2D Graphics Library.

ImageSharp Benchmark
SkiaSharp Benchmark

The initial comparison running on my development box (will benchmark on my Seeedstudio EdgeBox RPi 200.) was roughly what I was expecting though the SkaiSharp 2560×1440 mean duration was a bit odd. I think that the difference in the amount of memory allocated is because SkaiSharp’s memory is allocated by the native code. Both benchmarks need some refactoring to improve repeatability on my different platforms.

These benchmarks should be treated as indicative not authoritative