FohenAIBuilder Scan tool – 100% dog food

The first FoehnAIBuilder tool had to be a very basic scan tool so the LLM could find chunks of text in text files. The generated code had some issues which were pointed out by Copilot.

ReviewScanFiles.cs wonder how well is going to handle large files. I think more async would be good

Review: SearchFilesTool.cs — Large-File & Async Concerns

You’re right to be concerned. There are several places where large files or large trees will bite you.

Large-file / scalability issues

1. StreamReader with default buffer + no explicit encoding

new StreamReader(file) uses a 1 KB buffer and auto-detects encoding by reading a BOM. For large files this is inefficient. Pass a bigger buffer and make async real:

using var stream = new FileStream(
    file,
    FileMode.Open,
    FileAccess.Read,
    FileShare.ReadWrite | FileShare.Delete,
    bufferSize: 64 * 1024,
    FileOptions.Asynchronous | FileOptions.SequentialScan);
using var reader = new StreamReader(stream, detectEncodingFromByteOrderMarks: true);

FileOptions.SequentialScan hints the OS cache manager for forward-only reads. FileOptions.Asynchronous is what actually makes ReadLineAsync non-blocking — without it you’re doing sync I/O on the thread pool.

2. FileShare.Read (default) blocks files opened for write elsewhere

Locked files (log files, files being edited) throw and get skipped. FileShare.ReadWrite | FileShare.Delete lets you scan them.

3. Binary-file detection is weak

if (lineNumber == 1 && line.Contains('\0')) break;

Only checks line 1; many binaries have text at the top. And a huge binary with no \n will be read into memory as one giant string before this check fires. Better:

  • Read the first ~4–8 KB as bytes, look for \0 → skip if binary.
  • Or exclude common binary extensions up front (.dll, .exe, .pdb, .png, .jpg, .zip, .bin, .so, .dylib, etc.).

4. No per-file size cap

A 2 GB log file will be fully streamed. Add a guard:

var info = new FileInfo(file);
if (info.Length > MaxFileSizeBytes) continue;

5. No max-line-length guard

ReadLineAsync on a file with no newlines (minified JS, single-line log) will buffer the whole file into one string. Skip lines above ~8 KB.

6. matches stores full trimmed line

A minified match line could be megabytes. Truncate:

var snippet = line.Length > 500 ? line[..500] + "…" : line;
matches.Add((Path.GetRelativePath(fullPath, file), lineNumber, snippet.Trim()));

Async concerns

7. Sequential file iteration

The outer foreach awaits one file at a time. On an SSD scanning thousands of files, most time is spent waiting on I/O. Use Parallel.ForEachAsync with bounded concurrency:

var gate = new Lock(); // .NET 9+ Lock
await Parallel.ForEachAsync(
    files,
    new ParallelOptions
    {
        MaxDegreeOfParallelism = Environment.ProcessorCount,
        CancellationToken = cancellationToken
    },
    async (file, ct) =>
    {
        // scan file into a local list, then merge under `gate`
    });

Result order becomes non-deterministic — sort by (RelativePath, LineNumber) before printing so output is stable.

8. ReadLineAsync allocates a string per line

For hot paths, PipeReader + SequenceReader searching raw UTF-8 is much faster and lower-allocation, but more code. Probably overkill unless benchmarks say so.

9. Cancellation inside a huge file

ReadLineAsync(cancellationToken) honors cancellation on each read — good. The outer ThrowIfCancellationRequested becomes redundant once you parallelize (the ParallelOptions.CancellationToken covers it).

Other observations

  • filesScanned counter is misleading once parallelized — use Interlocked.Increment and only count successful opens.
  • sb.AppendLine uses Environment.NewLine. For LLM output, '\n' is more portable.
  • ToolPath.TryResolve(sandboxRoot, path, ...) — different signature than WriteFileTool uses. Worth checking consistency.
  • Skipped-file reporting: consider distinguishing filesScanned vs filesSkipped in the summary.

Suggested per-file scan method

private static async Task<List> ScanFileAsync(
    string file, string text, StringComparison comparison, int maxMatches, CancellationToken ct)
{
    var results = new List();

    var info = new FileInfo(file);
    if (info.Length == 0 || info.Length > 50L * 1024 * 1024) // 50 MB cap
        return results;

    await using var stream = new FileStream(
        file, FileMode.Open, FileAccess.Read,
        FileShare.ReadWrite | FileShare.Delete,
        bufferSize: 64 * 1024,
        FileOptions.Asynchronous | FileOptions.SequentialScan);

    // Cheap binary sniff on first 4 KB
    var sniffLen = (int)Math.Min(4096, info.Length);
    var sniff = new byte[sniffLen];
    var read = await stream.ReadAsync(sniff, ct);
    if (sniff.AsSpan(0, read).IndexOf((byte)0) >= 0) return results;
    stream.Position = 0;

    using var reader = new StreamReader(stream, detectEncodingFromByteOrderMarks: true);

    int lineNumber = 0;
    string? line;
    while ((line = await reader.ReadLineAsync(ct)) is not null)
    {
        lineNumber++;
        if (line.Length > 8192) continue; // skip absurdly long lines
        if (line.Contains(text, comparison))
        {
            var snippet = line.Length > 500 ? line[..500] + "…" : line;
            results.Add((lineNumber, snippet.Trim()));
            if (results.Count >= maxMatches) break;
        }
    }

    return results;
}

Priorities

  1. FileOptions.Asynchronous | FileOptions.SequentialScan + bigger buffer — biggest win, tiny change.
  2. Per-file size cap + better binary detection — prevents worst-case blow-ups.
  3. Snippet truncation — bounds memory of matches.
  4. Parallel.ForEachAsync — the real “more async” you asked for; scales with cores/SSD.
  5. FileShare.ReadWrite | FileShare.Delete — fewer spurious skips.

Want me to produce the full rewritten file?

The review wasn’t great but for a “nasty” PoC it will be okay. Need to work on exception catching and directory escape functionality.

FoehnAIBuilder Plug-in Tools Implementation

FoehnAIBuilder uses a plugin-based architecture where every tool implements a standard .NET class that conforms to the ITool C# interface. As part of the startup process, the application scans the plugin directory, loads all the available assemblies, and includes all that implement this contract.

namespace FoehnAIBuilder.Abstractions;

public interface ITool
{
    string Name { get; }
    string Description { get; }
    string Command { get; }

    ToolRiskLevel RiskLevel { get; }
    Task<ToolExecutionResult> ExecuteAsync(string argumentsJson, CancellationToken cancellationToken = default);
}

Tools also have a risk level (considering increasing the number of options and training an ML.NET model to detect potentially malicious arguments), so the host application can apply safety controls such as requiring user confirmation before operations that may have significant side effects.

public enum ToolRiskLevel
{
    Undefined = 0,
    ReadOnly,
    Write,
    Destructive,
}

FoehnAIBuilder enforces a maximum tool iteration count. This prevents runaway execution loops, stops the context growing to the point where it impacts on the LLM’s performance (The dumb zone), and “burning” lots of Tokens

   "Mistral": {
      "BaseUrl": "https://api.mistral.ai",
      "APIKey": "This is not the APIKey you are looking for",
      "DefaultModel": "devstral-latest",
      "TimeoutSeconds": 120,
      "MaxRetries": 3,
      "EnableStreaming": false
   },
   "FoehnAIBuilder": {
      "SystemMessageFile": "foehn.md",
      "PluginsPath": ".plugins",
      "WorkingDirectory": "",
      "MaxToolIterations": 30
   },
}

Each tool exposes metadata that allows the LLM to understand how to invoke it. This includes a unique function name, a human-readable description, and a JSON Schema describing the parameters the tool expects. I’m considering implementing the parameters for a plug-in using Data Transfer Objects (DTO) rather than the current approach using strings.

public string Name => "scan";

public string Description =>
    "Recursively lists files and directories under a given path, or the current working " +
    "folder if no path is supplied. Use this first to discover what exists before reading, " +
    "writing, deleting, or executing anything.";

public string Command => """
{
   "type": "object",
   "properties": {
        "path": { "type": "string", "description": "Directory to scan. Defaults to the application's current working folder if omitted." },
        "pattern": { "type": "string", "description": "Search pattern, e.g. '*.cs'. Defaults to '*' (all files)." },
         "recursive": { "type": "boolean", "description": "Whether to recurse into subdirectories. Defaults to true." }
    },
    "required": []
}

The plugins have code to detect an LLM directory escape with a path in a parameter like “directory to scan”. When the LLM chooses to invoke a tool, FoehnAIBuilder calls the tool’s ExecuteAsync method and passes the arguments as a JSON document that conforms to the schema exposed by the tool.

FoehnAIBuilder processes the request and returns a ToolExecutionResult, which provides a standardised way for both the application and the LLM to understand the outcome. The result contains a boolean success indicator and a descriptive message that may include returned data, status information, or error details.

public sealed class ToolExecutionResult
{
    public required bool Success { get; init; }

    public required string Result { get; init; }

    public static ToolExecutionResult Ok(string result) => new() { Success = true, Result = result };

    public static ToolExecutionResult Fail(string result) => new() { Success = false, Result = result };
}

ToolExecutionResult approach follows the result pattern rather than an exception-driven programming model. Every tool invocation returns a result containing both a success indicator and a human-readable message describing the outcome. This provides a consistent contract between the tool, the host, and the language model. This allows the LLM to reason about both successful operations and expected failure conditions such as validation errors, missing resources, or access restrictions.

The plug-in implementations handle and translate all anticipated error conditions into a ToolExecutionResult.Fail response rather than allowing exceptions to propagate to the FoehnAIBuilder host (this would be bad). Returning structured failure information enables the language model to understand what went wrong and potentially adjust its behaviour and retry with different inputs.

try
{
   var searchOption = recursive ? SearchOption.AllDirectories : SearchOption.TopDirectoryOnly;

   ...

   return Task.FromResult(ToolExecutionResult.Ok(sb.ToString()));
}
catch (Exception ex) when (ex is UnauthorizedAccessException or IOException)
{
   _logger.LogError(ex, "Error scanning {Path}", path);
   return Task.FromResult(ToolExecutionResult.Fail($"Error scanning \"{path}\": {ex.Message}"));
}

Exceptions are reserved for genuinely unexpected conditions such as programming errors, infrastructure failures, or unrecoverable runtime errors. As a general rule, no exception in a tool plug-in should be returned to FoehnAIBuilder for business logic or user-correctable error, these should always be represented as a failed ToolExecutionResult containing a clear and actionable explanation of the problem.

The next couple of posts will explore progressively more capable (read dangerous) operations. First, file and directory tools, where path traversal, deletion, and privilege boundaries (file and directory permissions) introduce real risk.

Mistral Chat Completions

Building a lightweight Codestral Chat Completion Command-Line Interface(CLI) is one of the fastest ways to understand how modern Large Language Model(LLM) Application Programming Interfaces (API) work in the real world. While everyone is talking about “agentic” programming (I now spend a lot of time reviewing code, as more agents = more reviews) and “token maxing” (difficult conversations with finance) this series of posts is about the plumbing.

I’m starting from the bottom and working my way up the stack. A raw Hypertext Transfer Protocol(HTTP) contract: an HTTP POST, a model name, a messages array, and a response object you have to parse yourself. With an HTTP proxy (Telerik Fiddler) I confirmed the Mistral Chat API endpoint Uniform Resource Locator(URL) and my API Key worked.

Before writing or generating typed Data Transfer Objects(DTO), or any kind of strongly‑typed client, streaming the response JSON into a jsonDocument was a good way to visualise the shape of the responses. I could enumerate properties, check for missing or inconsistent fields, validate casing, and confirm whether optional objects appear only in certain scenarios. Any undocumented polymorphic shapes, and response constructs can be impossible to model cleanly, and “hand-rolled” serialisation can be fragile.

//...
Console.Write("Enter chat message: ");
var prompt = Console.ReadLine();

// Anonymous type for the request body which feels bit "hinky" but, it works and is concise. Alternatively, could define a class for request body for better type safety and maintainability.
var requestObject = new
{
   model = settings.ModelName,
   messages = new[]
   {
      new { role = "user", content = prompt }
   }
};

// Alternatively, a JsonObject and JsonArray for more control over the JSON structure
var requestJson = new JsonObject()
{
   ["model"] = settings.ModelName,
   ["messages"] = new JsonArray
   {
      new JsonObject
      {
         ["role"] = "user",
         ["content"] = prompt
      }
   }
};

// Create HttpClient with required headers. Note that HttpClient should ideally be reused, but for simplicity we're creating a new instance here.
HttpClient httpClient = new()
{
   DefaultRequestHeaders =
   {
      Accept = { new MediaTypeWithQualityHeaderValue("application/json") },
      Authorization = new AuthenticationHeaderValue("Bearer", settings.ApiKey)
   },
   BaseAddress = new Uri(settings.BaseUrl)
};

using var httpResponse = await httpClient.PostAsync("chat/completions", new StringContent(JsonSerializer.Serialize(requestObject), Encoding.UTF8, "application/json"));

httpResponse.EnsureSuccessStatusCode();

using var stream = await httpResponse.Content.ReadAsStreamAsync();
using var responseDocument = await JsonDocument.ParseAsync(stream);

var content = responseDocument.RootElement.GetProperty("choices")[0].GetProperty("message").GetProperty("content").GetString();

Console.WriteLine(content);
Console.WriteLine();

var usage = responseDocument.RootElement.GetProperty("usage");
Console.WriteLine($"Prompt tokens: {usage.GetProperty("prompt_tokens").GetInt32()}");
Console.WriteLine($"Completion tokens: {usage.GetProperty("completion_tokens").GetInt32()}");
Console.WriteLine($"Total tokens: {usage.GetProperty("total_tokens").GetInt32()}");
Console.WriteLine();

Console.WriteLine("Press <Enter> to exit...");
Console.ReadLine();

Reading the response string and parsing it with a jsonDocument is straightforward but can be inefficient for large responses because it loads the entire response into memory.

I built the typed interface by inspecting real JSON responses especially structures like polymorphic message, and cross‑checking them against the Mistral API docs. This highlighted which fields are genuinely optional, which only appear for tool calls, and which vary by finish reason. It also highlighted that I would have to introduce polymorphic message classes so the interface can cleanly represent text messages, tool‑call messages, and whatever new variants the API adds later.

// Create HttpClient with required headers. Note that HttpClient should ideally be reused, but for simplicity we're creating a new instance here..
using HttpClient httpClient = new()
{
   DefaultRequestHeaders =
   {
      Accept = { new MediaTypeWithQualityHeaderValue("application/json") },
      Authorization = new AuthenticationHeaderValue("Bearer", settings.ApiKey)
   },
   BaseAddress = new Uri(settings.BaseUrl)
};

var jsonSerializerOptions = new JsonSerializerOptions()
{
   // PropertyNamingPolicy removed - [JsonPropertyName] attributes on model handle wire names
   WriteIndented           = false,
   DefaultIgnoreCondition  = JsonIgnoreCondition.WhenWritingNull,
   AllowTrailingCommas     = false,
   ReadCommentHandling     = JsonCommentHandling.Disallow,
   UnmappedMemberHandling  = JsonUnmappedMemberHandling.Skip,
};

Console.Write("Enter chat message: ");
var content = Console.ReadLine();

while (!string.IsNullOrWhiteSpace(content))
{
   var request = new ChatCompletionRequest
   {
      Model = settings.ModelName,
      Messages =
      [
         new ChatMessage { Role = "user", Content = content }
      ],
   };

   try
   {
      using var httpResponse = await httpClient.PostAsJsonAsync("chat/completions", request, jsonSerializerOptions);
      httpResponse.EnsureSuccessStatusCode();

      ChatCompletionResponse? chatCompletionResponse = await httpResponse.Content.ReadFromJsonAsync<ChatCompletionResponse>(jsonSerializerOptions);

      if (chatCompletionResponse != null)
      {
         foreach (var choice in chatCompletionResponse.Choices)
         {
            Console.WriteLine(choice.Message.Content);
         }

         Console.WriteLine();
         if (chatCompletionResponse.Usage != null)
         {
            Console.WriteLine($"Prompt tokens: {chatCompletionResponse.Usage.PromptTokens}");
            Console.WriteLine($"Completion tokens: {chatCompletionResponse.Usage.CompletionTokens}");
            Console.WriteLine($"Total tokens: {chatCompletionResponse.Usage.TotalTokens}");
         }
         Console.WriteLine();
      }
   }
   catch (HttpRequestException ex)
   {
      Console.WriteLine($"Request failed: {(int?)ex.StatusCode} {ex.Message}");
   }
   catch (TaskCanceledException)
   {
      Console.WriteLine("Request timed out.");
   }
   catch (JsonException ex)
   {
      Console.WriteLine($"Failed to parse response: {ex.Message}");
   }

   Console.Write("Enter chat message: ");
   content = Console.ReadLine();
}
public sealed class ChatCompletionRequest
{
   [JsonPropertyName("model")] public required string Model { get; init; }
   [JsonPropertyName("messages")] public required List<ChatMessage> Messages { get; init; }
   [JsonPropertyName("temperature")] public double? Temperature { get; init; }
   [JsonPropertyName("max_tokens")] public int? MaxTokens { get; init; }
   [JsonPropertyName("stream")] public bool? Stream { get; init; }
   [JsonPropertyName("response_format")] public ResponseFormat? ResponseFormat { get; init; }
}

public sealed class ChatMessage
{
   [JsonPropertyName("role")] public required string Role { get; init; }
   [JsonPropertyName("content")] public required string Content { get; init; }
}

public sealed class ResponseFormat
{
   [JsonPropertyName("type")] public required string Type { get; init; }
}

public sealed class ChatCompletionResponse
{
   [JsonPropertyName("id")] public required string Id { get; init; }
   [JsonPropertyName("choices")] public required List<ChatCompletionChoice> Choices { get; init; }
   [JsonPropertyName("usage")] public TokenUsage? Usage { get; init; }
}

public sealed class TokenUsage
{
   [JsonPropertyName("prompt_tokens")] public int? PromptTokens { get; init; }
   [JsonPropertyName("completion_tokens")] public int? CompletionTokens { get; init; }
   [JsonPropertyName("total_tokens")] public int? TotalTokens { get; init; }
}

public sealed class ChatCompletionChoice
{
   [JsonPropertyName("index")] public int Index { get; init; }
   [JsonPropertyName("message")] public required ChatMessage Message { get; init; }
   [JsonPropertyName("finish_reason")] public string? FinishReason { get; init; }
}

I had to capture the application output in two screenshots as the response text was longer.

The non-deterministic nature of LLMs resulted in different response messages, with the longest one consuming significantly more tokens, 530 vs. 799 (future posts will cover the use of Random_seed)