.NET nanoFramework RAK11200 – Brownout Voltage

My test setup was a RAK11200 WisBlock WiFi Module, RAK19001 WisBlock Base Board, RAK1901 WisBlock Temperature and Humidity Sensor, 1200mAH Lithium Polymer (LiPo) battery and SKU920100 Solar Board. The test setup uploads temperature, humidity and battery voltage telemetry to an Azure IoT Hub every 5 minutes (short delay so battery life reduced).

The first step was to check that I could get a “battery voltage” value for the RAKWireless RAK11200 WisBlock WiFi Module on a RAK19001 WisBlock Base Board for managing “brownouts” and send to my Azure IoT Hub.

RAK19001 Power supply schematic

The RAK19001 WisBlock Base Board has a voltage divider (R4&R5 with output ADC_VBAT) which is connected to pin 21(AIN0) on the CPU slot connector.

RAK19001 connector schematic

The RAK19001 WisBlock Base Board has quite a low leakage current so the majority of the power consumption should be the RAK11200 WisBlock WiFi Module.

RAK19001 leakage current from specifications

I used AdcController + AdcChannel to read AIN0 and modified the code using the formula (for a RAK4631 module) in the RAK Wireless forums to calculate the battery voltage. (UPDATE This calculation is not applicable to my scenario)

RAK11200 Schematic with battery voltage analog input highlighted

When “slept” the RAK11200 WisBlock WiFi Module power consumption is very low

RAK11200 low power current from specifications
public static void Main()
{
    Debug.WriteLine($"{DateTime.UtcNow:HH:mm:ss} devMobile.IoT.RAK.Wisblock.AzureIoTHub.RAK11200.PowerSleep starting");

    Thread.Sleep(5000); // This do debugger can attach consider removing in realease version

    try
    {
        double batteryVoltage;

        Configuration.SetPinFunction(Gpio.IO04, DeviceFunction.I2C1_DATA);
        Configuration.SetPinFunction(Gpio.IO05, DeviceFunction.I2C1_CLOCK);

        Debug.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Battery voltage measurement");

        // Configure Analog input (AIN0) port then read the "battery charge"
        AdcController adcController = new AdcController();

        using (AdcChannel batteryVoltageAdcChannel = adcController.OpenChannel(AdcControllerChannel))
        {

            // https://forum.rakwireless.com/t/custom-li-ion-battery-voltage-calculation-in-rak4630/4401/7
            // When I checked with multimeter I had to increase 1.72 to 1.9
            batteryVoltage = batteryVoltageAdcChannel.ReadValue() * (3.0 / 4096) * 1.9;

            Debug.WriteLine($" BatteryVoltage {batteryVoltage:F2}");

            if (batteryVoltage < Config.BatteryVoltageBrownOutThreshold)
            {
                Sleep.EnableWakeupByTimer(Config.FailureRetryInterval);
                Sleep.StartDeepSleep();
            }
        }

        Debug.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Wifi connecting");

        if (!WifiNetworkHelper.ConnectDhcp(Config.Ssid, Config.Password, requiresDateTime: true))
        {
            if (NetworkHelper.HelperException != null)
            {
                Debug.WriteLine($"{DateTime.UtcNow:HH:mm:ss} WifiNetworkHelper.ConnectDhcp failed {NetworkHelper.HelperException}");
            }

            Sleep.EnableWakeupByTimer(Config.FailureRetryInterval);
            Sleep.StartDeepSleep();
        }
        Debug.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Wifi connected");

        // Configure the SHTC3 
        I2cConnectionSettings settings = new(I2cDeviceBusID, Shtc3.DefaultI2cAddress);

        string payload ;

        using (I2cDevice device = I2cDevice.Create(settings))
        using (Shtc3 shtc3 = new(device))
        {
            if (shtc3.TryGetTemperatureAndHumidity(out var temperature, out var relativeHumidity))
            {
                Debug.WriteLine($" Temperature {temperature.DegreesCelsius:F1}°C Humidity {relativeHumidity.Value:F0}% BatteryVoltage {batteryVoltage:F2}");

                payload = $"{{\"RelativeHumidity\":{relativeHumidity.Value:F0},\"Temperature\":{temperature.DegreesCelsius:F1}, \"BatteryVoltage\":{batteryVoltage:F2}}}";
            }
            else
            {
                Debug.WriteLine($" BatteryVoltage {batteryVoltage:F2}");

                payload = $"{{\"BatteryVoltage\":{batteryVoltage:F2}}}";
            }

#if SLEEP_SHT3C
            shtc3.Sleep();
#endif
        }

        // Configure the HttpClient uri, certificate, and authorization
        string uri = $"{Config.AzureIoTHubHostName}.azure-devices.net/devices/{Config.DeviceID}";

        HttpClient httpClient = new HttpClient()
        {
            SslProtocols = System.Net.Security.SslProtocols.Tls12,
            HttpsAuthentCert = new X509Certificate(Config.DigiCertBaltimoreCyberTrustRoot),
            BaseAddress = new Uri($"https://{uri}/messages/events?api-version=2020-03-13"),
        };
        httpClient.DefaultRequestHeaders.Add("Authorization", SasTokenGenerate(uri, Config.Key, DateTime.UtcNow.Add(Config.SasTokenRenewFor)));

        Debug.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Azure IoT Hub device {Config.DeviceID} telemetry update start");

        HttpResponseMessage response = httpClient.Post("", new StringContent(payload));

        Debug.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Response code:{response.StatusCode}");

        response.EnsureSuccessStatusCode();
    }
    catch (Exception ex)
    {
        Debug.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Azure IoT Hub telemetry update failed:{ex.Message} {ex?.InnerException?.Message}");

        Sleep.EnableWakeupByTimer(Config.FailureRetryInterval);
        Sleep.StartDeepSleep();
    }

    Sleep.EnableWakeupByTimer(Config.TelemetryUploadInterval);
#if SLEEP_LIGHT
    Sleep.StartLightSleep();
#endif
#if SLEEP_DEEP
    Sleep.StartDeepSleep();
#endif
}

The nanoFramework.Hardware.Esp32.Sleep functionality supports LightSleep and DeepSleep states. The ESP32 device can be “woken up” by GPIO pin(s), Touch pad activity or by a Timer.

RAK11200+RAK19007+RAK1901+ LiPo battery test rig

After some “tinkering” I found the voltage calculation was surprisingly accurate (usually within 0.01V) for my RAK19001 and RAK19007 base boards.

When the battery voltage was close to its minimum working voltage of the ESP32 device it would reboot when the WifiNetworkHelper.ConnectDhcp method was called. This would quickly drain the battery flat even when the solar panel was trying to charge the battery.

Now, before trying to connect to the wireless network the battery voltage is checked and if too low (more experimentation required) the device goes into a deep sleep for a configurable period (more experimentation required). This is so the solar panel can charge the battery to a level where wireless connectivity will work.

.NET nanoFramework SHT20 library on Github

The full source code (just need to do readme) of my .NET nanoFramework Sensirion SHT20 temperature and humidity(Waterproof) library is now available on GitHub. I have tested the library and sample application with Sparkfun Thing Plus and ST Micro STM32F7691 Discovery devices. (I can validate on more platform configurations if there is interest).

Important: make sure you setup the I2C pins especially on ESP32 Devices before creating the I2cDevice,

SHT20 +STM32F769 Discovery test rig

The .NET nanoFramework device libraries use a TryGet… pattern to retrieve sensor value, this library throws an exception if reading a sensor value fails. I’m not certain which approach is “better” as reading Sensirion SHT20 temperature and humidity(Waterproof) has never failed The only time reading a value failed was when I unplugged the device which I think is “exceptional”.

//---------------------------------------------------------------------------------
// Copyright (c) March 2023, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// nanoff --target ST_STM32F769I_DISCOVERY --update 
// nanoff --platform ESP32 --serialport COM7 --update
//
//---------------------------------------------------------------------------------
#define ST_STM32F769I_DISCOVERY 
//#define  SPARKFUN_ESP32_THING_PLUS
namespace devMobile.IoT.Device.Sht20
{
    using System;
    using System.Device.I2c;
    using System.Threading;

#if SPARKFUN_ESP32_THING_PLUS
    using nanoFramework.Hardware.Esp32;
#endif

    class Program
    {
        static void Main(string[] args)
        {
            const int busId = 1;

            Thread.Sleep(5000);

#if SPARKFUN_ESP32_THING_PLUS
            Configuration.SetPinFunction(Gpio.IO23, DeviceFunction.I2C1_DATA);
            Configuration.SetPinFunction(Gpio.IO22, DeviceFunction.I2C1_CLOCK);
#endif

            I2cConnectionSettings i2cConnectionSettings = new(busId, Sht20.DefaultI2cAddress);

            using I2cDevice i2cDevice = I2cDevice.Create(i2cConnectionSettings);
            {
                using (Sht20 sht20 = new Sht20(i2cDevice))
                {
                    sht20.Reset();

                    while (true)
                    {
                        double temperature = sht20.Temperature();
                        double humidity = sht20.Humidity();
#if HEATER_ON_OFF
					    sht20.HeaterOn();
					    Console.WriteLine($"{DateTime.Now:HH:mm:ss} HeaterOn:{sht20.IsHeaterOn()}");
#endif
                        Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Temperature:{temperature:F1}°C Humidity:{humidity:F0}% HeaterOn:{sht20.IsHeaterOn()}");
#if HEATER_ON_OFF
					    sht20.HeaterOff();
					    Console.WriteLine($"{DateTime.Now:HH:mm:ss} HeaterOn:{sht20.IsHeaterOn()}");
#endif
                        Thread.Sleep(1000);
                    }
                }
            }
        }
    }
}

I’m going to soak test the library for a week to check that is working okay, then most probably refactor the code so it can be added to the nanoFramework IoT.Device Library repository.

.NET nanoFramework Seeedstudio HM3301 Basic connectivity

This is a “throw away” .NET nanoFramework application for investigating how Seeedstudio Grove HM3301 Inter Integrated Circuit bus(I²C) connectivity works.

Seeedstudio Grove HM3301 Sensor

My test setup is a simple .NET nanoFramework console application running on an STM32F7691 Discovery board.

Seeedstudio Grove HM3301 + STM32F769 Discovery test rig

The HM3301I2C application has lots of magic numbers from the HM3301 datasheet and is just a tool for exploring how the sensor works.

public static void Main()
{
    I2cConnectionSettings i2cConnectionSettings = new(1, 0x40);

    // i2cDevice.Dispose
    I2cDevice i2cDevice = I2cDevice.Create(i2cConnectionSettings);

    while (true)
    {
        byte[] writeBuffer = new byte[1];
        byte[] readBuffer = new byte[29];

        writeBuffer[0] = 0x88;

        i2cDevice.WriteRead(writeBuffer, readBuffer);

        //i2cDevice.WriteByte(0x88);
        //i2cDevice.Read(readBuffer);

        ushort standardParticulatePm1 = (ushort)(readBuffer[4] << 8);
        standardParticulatePm1 |= readBuffer[5];

        ushort standardParticulatePm25 = (ushort)(readBuffer[6] << 8);
        standardParticulatePm25 |= readBuffer[7];

        ushort standardParticulatePm10 = (ushort)(readBuffer[8] << 8);
                standardParticulatePm10 |= readBuffer[9];

        Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Standard particulate    PM 1.0: {standardParticulatePm1}  PM 2.5: {standardParticulatePm25}  PM 10.0: {standardParticulatePm10} ug/m3");

        ushort atmosphericPm1 = (ushort)(readBuffer[10] << 8);
        atmosphericPm1 |= readBuffer[11];

        ushort atmosphericPm25 = (ushort)(readBuffer[12] << 8);
        atmosphericPm25 |= readBuffer[13];

        ushort atmosphericPm10 = (ushort)(readBuffer[14] << 8);
        atmosphericPm10 |= readBuffer[15];

        Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Atmospheric particulate PM 1.0: {atmosphericPm1:3}  PM 2.5: {atmosphericPm25}  PM 10.0: {atmosphericPm10} ug/m3");


        ushort particulateCountPm03 = (ushort)(readBuffer[16] << 8);
        particulateCountPm03 |= readBuffer[17];

        ushort particulateCountPm05 = (ushort)(readBuffer[18] << 8);
        particulateCountPm05 |= readBuffer[19];

        ushort particulateCountPm1 = (ushort)(readBuffer[20] << 8);
        particulateCountPm1 |= readBuffer[21];

        Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Particulate count       PM 0.3: {particulateCountPm03:3}  PM 0.5: {particulateCountPm05}  PM 1.0: {particulateCountPm1} ug/m3");


        ushort particleCountPm25 = (ushort)(readBuffer[22] << 8);
        particleCountPm25 |= readBuffer[23];

        ushort particleCountPm5 = (ushort)(readBuffer[24] << 8);
        particleCountPm5 |= readBuffer[25];

        ushort particleCountPm10 = (ushort)(readBuffer[26] << 8);
        particleCountPm10 |= readBuffer[27];

        Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Particle count/0.1L     PM 2.5: {particleCountPm25}  PM 5.0: {particleCountPm5}  PM 10.0: {particleCountPm10} particles/0.1L");


        byte checksum = 0;
        for (int i = 0; i < readBuffer.Length - 1; i++)
        {
            checksum += readBuffer[i];
        }
        Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Checksum payload:{readBuffer[28]} calculated:{checksum}");
        Console.WriteLine("");

        Thread.Sleep(5000);
    }
}

The unpacking of the value standard particulate, particulate count and particle count values is fairly repetitive, but I will fix it in the next version.

Visual Studio 2022 Debug output

The checksum calculation isn’t great even a simple cyclic redundancy check(CRC) would be an improvement on summing the 28 bytes of the payload.

.NET nanoFramework Qorvo DW1000 – RAK13801 Device SPI

When developing libraries it’s good to have a selection of different platforms for testing as this can significantly improve the quality and robustness of the implementation. A few months ago I noticed that RAK Wireless have a UWB Module Decawave DW1000 Wisblock so I added one to an order.

My second Qorvo DW1000 setup is a RAK120000 Wisblock Core module, on a RAK19007 WisBlock Base with a RAK13801 WisBlock Wireless module

RAK12000 + RAK19007 + RAK13801 test platform

The Qorvo DW1000 module has a Serial Peripheral Interface (SPI) so the Master In Slave Out(MISO), Master Out Slave In(MOSI), Serial Clock(SCLK) and Chip Slave Select(CSS) pins of the RAK11200 WisBiock Core Module have to be setup using the Configuration.SetPinFunction method of the nanoFramework.Hardware.Esp32 library.

RAK11200 Schematic with SPI pins highlighted.
RAK13801 Schematic with SPI pins highlighted.

I have added a couple of C# processor directives (MAKERFABS_ESP32UWB & RAK11200_RAK1907_RAK13801) so the platform that the Qorvo DW1000 module is running on can be configured.

public class Program
{
#if MAKERFABS_ESP32UWB
    private const int SpiBusId = 1;
    private const int chipSelectLine = Gpio.IO04;
#endif
#if RAK11200_RAK1907_RAK13801
    private const int SpiBusId = 1;
    private const int chipSelectLine = Gpio.IO32;
#endif

    public static void Main()
    {
        Thread.Sleep(5000);

        Debug.WriteLine("devMobile.IoT.Dw1000.ShieldSPI starting");

        try
        {
#if MAKERFABS_ESP32UWB
            Configuration.SetPinFunction(Gpio.IO19, DeviceFunction.SPI1_MISO);
            Configuration.SetPinFunction(Gpio.IO23, DeviceFunction.SPI1_MOSI);
            Configuration.SetPinFunction(Gpio.IO18, DeviceFunction.SPI1_CLOCK);
#endif
#if RAK11200_RAK1907_RAK13801
            Configuration.SetPinFunction(Gpio.IO35, DeviceFunction.SPI1_MISO);
            Configuration.SetPinFunction(Gpio.IO25, DeviceFunction.SPI1_MOSI);
            Configuration.SetPinFunction(Gpio.IO33, DeviceFunction.SPI1_CLOCK);
#endif
            var settings = new SpiConnectionSettings(SpiBusId, chipSelectLine)
            {
                ClockFrequency = 2000000,
                Mode = SpiMode.Mode0,
            };

            using (SpiDevice device = SpiDevice.Create(settings))
            {
                while (true)
                {
                    byte[] writeBuffer = new byte[] { 0x0, 0x0, 0x0, 0x0, 0x0 }; // 0x0 = DEV_ID
                    byte[] readBuffer = new byte[writeBuffer.Length];

                    device.TransferFullDuplex(writeBuffer, readBuffer); // 15, 48, 1, 202, 222

                    uint ridTag = (uint)(readBuffer[4]<< 8 | readBuffer[3]);
                    byte model = readBuffer[2];
                    byte ver = (byte)(readBuffer[1] >> 4);
                    byte rev = (byte)(readBuffer[1] & 0x0f);

                    Debug.WriteLine(String.Format($"RIDTAG 0x{ridTag:X2} MODEL 0x{model:X2} VER 0X{ver:X2} REV 0x{rev:X2}"));

                   Thread.Sleep(10000);
                }
            }
        }
        catch (Exception ex)
        {
            Debug.WriteLine(ex.Message);
        }
    }
}

The alignment of the RAK11200 WisBiock Core Module pins and labels on the circuit diagram tripped me up. My initial configuration caused the device to reboot every time the application started.

Visual Studio 2022 Debug window displaying the decoded value from Register 0x0

At the top of test applications, I usually have a brief delay i.e Thread.Sleep(5000) so I can attach the debugger or erase the flash before the application crashes.

.NET nanoFramework Qorvo DW1000 – Makerfabs Device SPI

The Makerfabs ESP32 UWB(Ultra Wideband) module has a Qorvo DW1000 and Espressif ESP32 module. The Espressif ESP32 module can run the .NET nanoFramework but does not have a Qorvo DW1000 library. (March2023)

Makerfabs ESP32 UWB(Ultra Wide Band) module

Before any coding I used nanoff to “flash” the Espressif ESP32 module with the latest version of .NET nanoFramework

Flashing Makerfabs ESP32 UWB module with nanoff

The Qorvo DW1000 module has a Serial Peripheral Interface (SPI) so the Master In Slave Out(MISO), Master Out Slave In(MOSI), Serial Clock(SCLK) and Chip Slave Select(CSS) pins have to be configured using the Configuration.SetPinFunction method of the nanoFramework.Hadware.Esp32 library

Makerfabs ESP32 UWB module schematic

Even though SPI is an industry standard there are often subtle differences which need to be taken into account when reading from/writing to registers. The DW1000 has a static “Device Identifier” which I used to debug my “proof of concept” code.

DW1000 Datasheet Register Map documentation for Register 0x00

The DeviceSPI program reads register 0x00 and then displays the decoded payload.

public class Program
{
#if MAKERFABS_ESP32UWB
    private const int SpiBusId = 1;
    private const int chipSelectLine = Gpio.IO04;
#endif

    public static void Main()
    {
        Thread.Sleep(5000);

        Debug.WriteLine("devMobile.IoT.Dw1000.ShieldSPI starting");

        try
        {
#if MAKERFABS_ESP32UWB
            Configuration.SetPinFunction(Gpio.IO19, DeviceFunction.SPI1_MISO);
            Configuration.SetPinFunction(Gpio.IO23, DeviceFunction.SPI1_MOSI);
            Configuration.SetPinFunction(Gpio.IO18, DeviceFunction.SPI1_CLOCK);
#endif
            var settings = new SpiConnectionSettings(SpiBusId, chipSelectLine)
            {
                ClockFrequency = 2000000,
                Mode = SpiMode.Mode0,
            };

            using (SpiDevice device = SpiDevice.Create(settings))
            {
                Thread.Sleep(500);

                while (true)
                {
                    /*
                    byte[] writeBuffer = new byte[] { 0x0, 0x0, 0x0, 0x0, 0x0 }; // 0x0 = DEV_ID
                    byte[] readBuffer = new byte[writeBuffer.Length];

                    device.TransferFullDuplex(writeBuffer, readBuffer); // 15, 48, 1, 202, 222
                    */

                    byte[] writeBuffer = new byte[] { 0x0 }; // 0x0 = DEV_ID
                    byte[] readBuffer = new byte[5];

                    device.TransferFullDuplex(writeBuffer, readBuffer); // 15, 48, 1, 202, 222
                       
                    uint ridTag = (uint)(readBuffer[4]<< 8 | readBuffer[3]);
                    byte model = readBuffer[2];
                    byte ver = (byte)(readBuffer[1] >> 4);
                    byte rev = (byte)(readBuffer[1] & 0x0f);

                    Debug.WriteLine(String.Format($"RIDTAG 0x{ridTag:X2} MODEL 0x{model:X2} VER 0X{ver:X2} REV 0x{rev:X2}"));

                    Thread.Sleep(10000);
                }
            }
        }
        catch (Exception ex)
        {
            Debug.WriteLine(ex.Message);
        }
    }
}
Visual Studio 2022 Debug window displaying the decoded value from Register 0x0

The DW1000 User Manual is > 240 pages, with roughly 140 pages of detailed documentation about the DW1000 register set so progress will be slow.

.NET nanoFramework SHT20 Basic connectivity

A couple of years ago I wrote a .NET Core library for the Sensirion SHT20 temperature and humidity(Waterproof) sensor from DFRobot. This .NET nanoFramework version was “inspired” by the .NET Core library version, though I have added some message validation functionality.

DF Robot SHT20 Waterproof sensor

My test setup is a simple .NET nanoFramework console application running on an STM32F7691 Discovery board.

Discovery STM32F769 + SHT20 Testrig

The SH20DeviceI2C application has lots of magic numbers from the SHT20 datasheet and was just a tool for exploring how the sensor works.

 public static void Main()
{
    I2cConnectionSettings i2cConnectionSettings = new(1, 0x40);

    // i2cDevice.Dispose in final program
    I2cDevice i2cDevice = I2cDevice.Create(i2cConnectionSettings);

    while (true)
    {
        byte[] readBuffer = new byte[3] { 0, 0, 0 };

        // First temperature
        i2cDevice.WriteByte(0xF3);

        //Thread.Sleep(50); // no go -46.8
        //Thread.Sleep(60);
        Thread.Sleep(70);
        //Thread.Sleep(90);
        //Thread.Sleep(110);

        i2cDevice.Read(readBuffer);

        ushort temperatureRaw = (ushort)(readBuffer[0] << 8);
        temperatureRaw += readBuffer[1];

        //Debug.WriteLine($"Raw {temperatureRaw}");

        double temperature = temperatureRaw * (175.72 / 65536.0) - 46.85;

        // Then read the Humidity
        i2cDevice.WriteByte(0xF5);

        //Thread.Sleep(50);  
        //Thread.Sleep(60);  
        Thread.Sleep(70);  
        //Thread.Sleep(90);  
        //Thread.Sleep(110);   
                
        i2cDevice.Read(readBuffer);

        ushort humidityRaw = (ushort)(readBuffer[0] << 8);
        humidityRaw += readBuffer[1];

        //Debug.WriteLine($"Raw {humidityRaw}");

        double humidity = humidityRaw * (125.0 / 65536.0) - 6.0;

        //Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Temperature:{temperature:F1}°C");
        //Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Humidity:{humidity:F0}%");
        Console.WriteLine($"{DateTime.UtcNow:HH:mm:ss} Temperature:{temperature:F1}°C Humidity:{humidity:F0}%");

        Thread.Sleep(1000);
    }
}

While tinkering with the sensor I found that having a short delay between initiating the temperature reading (TemperatureNoHold = 0xF3 was used so as not to hang up the I2C bus) and reading the value was important.

Temperature value without Thread.Sleep

When I ran the application without a Thread.Sleep(70) the temperature and/or humidity the values were incorrect and sometimes quite random.

Temperature value with Thread.Sleep(70)
Humidity value without Thread.Sleep
Humidity value with Thread.Sleep(70)
Temperature and Humidity values with Thread.Sleep(70)

The .NET Core library didn’t validate the message payload Cyclic Redundancy Check (CRC) so I have added that in this version

void CheckCrc(byte[] bytes, byte bytesLen, byte checksum)
{
    var crc = 0;

    for (var i = 0; i < bytesLen; i++)
    {
        crc ^= bytes[i];
        for (var bit = 8; bit > 0; --bit)
        {
            crc = ((crc & 0x80) == 0x80) ? ((crc << 1) ^ CrcPolynomial) : (crc << 1);
        }
    }

    if (crc != checksum)
    {
        throw new Exception("CRC Error");
    }
}

The CheckCrc is called in Temperature and Humidity methods.

public double Temperature()
{
    byte[] readBuffer = new byte[3] { 0, 0, 0 };
    if (_i2cDevice == null)
    {
        throw new ArgumentNullException(nameof(_i2cDevice));
    }

    _i2cDevice.WriteByte(TemperatureNoHold);

    Thread.Sleep(ReadingWaitmSec);

    _i2cDevice.Read(readBuffer);

    CheckCrc(readBuffer, 2, readBuffer[2]);

    ushort temperatureRaw = (ushort)(readBuffer[0] << 8);
    temperatureRaw += readBuffer[1];

    double temperature = temperatureRaw * (175.72 / 65536.0) - 46.85;

    return temperature;
}

I’m going to soak test the library for a week to check that is working okay, then refactor the code so it can be added to the nanoFramework IoT.Device Library repository.

Azure Functions Isolated Worker support for VB.Net 4.8

As part of my “day job” I spend a bit of time working with VB.Net 4.X “legacy” projects doing upgrades, and bug fixes. Currently I am updating a number of Windows Service applications to run as Microsoft Azure Functions. With the release of the Azure functions runtime V4 Isolated Worker Processes with .NET Framework V4.8 support this is the last post in my Azure Functions with VB.Net 4.X and Azure Functions with VB.Net on .NET Core V6 series.

I have published source code for Azure Storage BlobTrigger, Azure Storage QueueTrigger, and TimerTriggers.

Visual Studio Solution explorer Azure Functions projects

All of the examples now have a program.vb which initialises the Trigger.

Namespace VBNet....TriggerIsolated
    Friend Class Program
        Public Shared Sub Main(ByVal args As String())
            Call FunctionsDebugger.Enable()

            Dim host = New HostBuilder().ConfigureFunctionsWorkerDefaults().Build()

            host.Run()
        End Sub
    End Class
End Namespace

All of the Isolated worker process Triggers displayed this message which appeared to be benign.

Csproj not found in C:\Users\..\VBNetHttpTriggerIsolated\bin\Debug\net48 directory tree. Skipping user secrets file configuration.

There were a lot of articles about problems building Docker images but the only relevant ones appeared to talk about getting F# and other .NET Core languages to work in Azure Functions.

Namespace devMobile.Azure.VBNetBlobTriggerIsolated
    Public Class BlobTrigger
        Private ReadOnly _logger As ILogger

        Public Sub New(ByVal loggerFactory As ILoggerFactory)
            _logger = loggerFactory.CreateLogger(Of BlobTrigger)()
        End Sub

        <[Function]("vbnetblobtriggerisolated")>
        Public Sub Run(
        <BlobTrigger("vbnetblobtriggerisolated/{name}", Connection:="blobendpoint")> ByVal myBlob As String, ByVal name As String)

            _logger.LogInformation($"VB.Net NET 4.8 Isolated Blob trigger function Processed blob Name: {name}  Data: {myBlob}")
        End Sub
    End Class
End Namespace

I used Azure Storage Explorer to upload files containing Lorem Ipsum for testing the BlobTrigger.

Azure BlobTrigger function running in the desktop emulator
Azure BlobTrigger Function logging in Application Insights

I used Telerik Fiddler to POST messages to the desktop emulator and Azure endpoints.

Namespace VBNetHttpTriggerIsolated
    Public Class HttpTrigger
        Private Shared executionCount As Int32
        Private ReadOnly _logger As ILogger

        Public Sub New(ByVal loggerFactory As ILoggerFactory)
            _logger = loggerFactory.CreateLogger(Of HttpTrigger)()
        End Sub

        <[Function]("Notifications")>
        Public Function Run(
        <HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")> ByVal req As HttpRequestData) As HttpResponseData
            Interlocked.Increment(executionCount)

            _logger.LogInformation("VB.Net NET 4.8 Isolated HTTP trigger Execution count:{executionCount} Method:{req.Method}", executionCount, req.Method)

            Dim response = req.CreateResponse(HttpStatusCode.OK)
            response.Headers.Add("Content-Type", "text/plain; charset=utf-8")

            Return response
        End Function
    End Class
End Namespace
Azure HttpTrigger Function running in the desktop emulator
Azure HttpTrigger Function logging in Application Insights

I used Azure Storage Explorer to create messages for testing the QueueTrigger

Namespace devMobile.Azure.VBNetQueueTriggerIsolated

    Public Class QueueTrigger
        Private Shared _logger As ILogger
        Private Shared _concurrencyCount As Integer = 0
        Private Shared _executionCount As Integer = 0

        Public Sub New(ByVal loggerFactory As ILoggerFactory)
            _logger = loggerFactory.CreateLogger(Of QueueTrigger)()
        End Sub

        <[Function]("VBNetQueueTriggerIsolated")>
        Public Sub Run(
        <QueueTrigger("vbnetqueuetriggerisolated", Connection:="QueueEndpoint")> ByVal message As String)
            Interlocked.Increment(_concurrencyCount)
            Interlocked.Increment(_executionCount)

            _logger.LogInformation("VB.Net .NET 4.8 Isolated Queue Trigger Concurrency:{_concurrencyCount} ExecutionCount:{_executionCount} Message:{message}", _concurrencyCount, _executionCount, message)

            Interlocked.Decrement(_concurrencyCount)
        End Sub
    End Class
End Namespace
Azure QueueTrigger Function running in the desktop emulator
Azure QueueTrigger Function logging in Application Insights
Namespace devMobile.Azure.VBNetTimerTriggerIsolated
    Public Class TimerTrigger
        Private Shared _logger As ILogger
        Private Shared _executionCount As Integer = 0

        Public Sub New(ByVal loggerFactory As ILoggerFactory)
            _logger = loggerFactory.CreateLogger(Of TimerTrigger)()
        End Sub

        <[Function]("Timer")>
        Public Sub Run(
        <TimerTrigger("0 */1 * * * *")> ByVal myTimer As MyInfo)

            Interlocked.Increment(_executionCount)
            _logger.LogInformation("VB.Net Isolated TimerTrigger next trigger:{0} Execution count:{1}", myTimer.ScheduleStatus.Next, _executionCount)
        End Sub
    End Class
Azure TimerTrigger Function running in the desktop emulator
Azure TimerTrigger Function logging in Application Insights

The development, debugging and deployment of these functions took a lot of time. Initially Azure Application Insights didn’t work when the Azure Isolated Worker triggers were deployed to Azure. After some experimentation I found that Application Insights Connection Strings worked and Application Instrumentation Keys did not.

With the Microsoft: ‘We Do Not Plan to Evolve Visual Basic as a Language this should hopefully be my last post about VB.Net ever.

Azure Functions with VB.Net on .NET Core V6

A year and a half ago I wrote a post about how to build Azure functions with VB.Net and the .NET Framework 4.X. The Microsoft VB team posted about Visual Basic Support for .NET 5.0 in March 2020 then went quiet, so my customer put the project on hold. Since then, a lot has changed .NET Core 3.1 LTS ends December 12, 2022, and .NET Core 5.0 support (no LTS) ended May 10, 2022 so I have ported the samples to .NET Core V6.

The process is similar (but different) to the original approach

The VB.Net Solution from June 2021

First step is to create a Visual Basic .NET Core V6 console application

Visual Studio 2022 “Add a new project”

The specify a name for the new project.

Visual Studio 2022 Add Project “Configure your new project”

Then select the version of .NET Core used

Visual Studio 2022 Add Project “Additional information”

Then rename program.cs to a name which highlights that it is a trigger

Visual Studio 2022 rename program.vb to TimerTrigger.vb

The initial version of the TimerTrigger code was “inspired” by the VB.Net 4.8 version.

'---------------------------------------------------------------------------------
' Copyright (c) November 2022, devMobile Software
'
' Licensed under the Apache License, Version 2.0 (the "License");
' you may Not use this file except in compliance with the License.
' You may obtain a copy of the License at
'
'     http://www.apache.org/licenses/LICENSE-2.0
'
' Unless required by applicable law Or agreed to in writing, software
' distributed under the License Is distributed on an "AS IS" BASIS,
' WITHOUT WARRANTIES Or CONDITIONS OF ANY KIND, either express Or implied.
' See the License for the specific language governing permissions And
' limitations under the License.
'
'---------------------------------------------------------------------------------
Imports System.Threading

Imports Microsoft.Azure.WebJobs
Imports Microsoft.Extensions.Logging


Public Class TimerTrigger
    Shared executionCount As Int32

    <FunctionName("Timer")>
    Public Shared Sub Run(<TimerTrigger("0 */1 * * * *")> myTimer As TimerInfo, log As ILogger)
        Interlocked.Increment(executionCount)

        log.LogInformation("VB.Net .NET V6 TimerTrigger next trigger:{0} Execution count:{1}", myTimer.ScheduleStatus.Next, executionCount)

    End Sub
End Class

Visual Studio 2022 highlighting missing libraries
Visual Studio 2022 with additional function SDK references

The next step is to add the hosts.json(empty for timer tigger) and localsettings.json to configure the function

Visual 2022 Hosts.json file
Visual Studio 2022 showing hosts.json & local.settings.json

Then I could run the function in the Azure Functions runtime emulator and “single step” in the Visual Studio 2022 Debugger.

VB.Net .NET Core V6 Timer Trigger running in emulator

For completeness I also built sample BlobTrigger, HttpTrigger and QueueTrigger versions

VB.Net .NET Core V6 Blob Trigger running in emulator
VB.Net .NET Core V6 HTTP Trigger running in emulator
VB.Net .NET Core V6 Queue Trigger running in emulator

I also deployed the Azure Storage QueueTrigger to Microsoft Azure, configured it, and then stress tested it with multiple instances of my QueueMessageGenerator.

Queue Trigger Function deployment
Queue Trigger configuration
Queue Trigger Throughput 48K messages

What if it goes wrong…

“Can’t determine project language from files. Please add one of [–csharp, –javascript, –typescript, –java, –powershell, –customer]

Check “FUNCTIONS_WORKER_RUNTIME” in the local.settings.json file.

The baked in error logging doesn’t handle broken message formats very well. Look at the call stack or single step through the application to find the message format that is broken

Visual Studio 2022 editor with malformed message highlighted

WARNING

I assume this is not a supported approach so use

“at your own risk”

.NET Core web API + Dapper – Authorisation of Data Access

The theme of this post is controlling users’ ability to read and write rows in a table. The best scenario I could come up with using the World Wide Importers database was around controlling access to Customer information.

This would be a representative set of “project requirements”…

  • Salespeople tend to look after categories of Customers
    • Kayla – Novelty Shops
    • Hudson – Supermarkets
    • Issabella – Computer Stores
    • Sophia – Gift Stores, Novelty Shops
    • Amy – Corporates
    • Anthony – Novelty Stores
    • Alica – Coporates
    • Stella – Supermarkets
  • But some Salespeople have direct relationships with Customers
    • Kayla – Corporate customers Eric Torres & Cosmina
    • Hudson – Tailspin Toys Head Office
    • Issabell – Tailspin Toys (Sylvanite, MT), Tailspin Toys (Sun River, MT), Tailspin Toys (Sylvanite, MT)
  • No changes to the database which could break the existing solution

In a previous engagement we added CustomerCategoryPerson and CustomerPerson like tables to the database to control read/write access to Customers’ information.

The CustomerCategoryPerson table links the CustomerCategory and Person tables with a flag (IsWritable) which indicates whether the Person can read/write Customer information for all the Customers in a CustomerCategory.

CREATE TABLE [Sales].[CustomerCategoryPerson](
	[CustomerCategoryPersonID] [int] IDENTITY(1,1) NOT NULL,
	[CustomerCategoryId] [int] NOT NULL,
	[PersonId] [int] NOT NULL,
	[IsWritable] [bit] NOT NULL,
	[LastUpdatedBy] [int] NOT NULL,
 CONSTRAINT [PK_CustomerCategoryPerson] PRIMARY KEY CLUSTERED 
(
	[CustomerCategoryPersonID] ASC
)...

The CustomerPerson table links the Customer and Person tables with a flag (IsWritable) which indicates whether a Person can read/write a Customer’s information.

CREATE TABLE [Sales].[CustomerPerson](
	[CustomerPersonId] [int] IDENTITY(1,1) NOT NULL,
	[CustomerID] [int] NOT NULL,
	[PersonId] [int] NOT NULL,
	[IsWritable] [bit] NOT NULL,
	[LastEditedBy] [int] NOT NULL,
 CONSTRAINT [PK_CustomerPerson] PRIMARY KEY CLUSTERED 
(
	[CustomerPersonId] ASC
)...

Users can do “wildcard” searches for Customers and the results set has to be limited to “their” customers and customers in the customer categories they are assigned too.

ALTER PROCEDURE [Sales].[CustomersNameSearchUnionV1]
@UserId as int,
@SearchText nvarchar(20),
@MaximumRowsToReturn int
AS
BEGIN
	-- Individual assignment
    SELECT TOP(@MaximumRowsToReturn) [Customers].[CustomerID] as "ID", [Customers].[CustomerName] as "Name", [Customers].[IsOnCreditHold] as "IsOnCreditHold"
    FROM Sales.Customers
	INNER JOIN [Sales].[CustomerPerson] ON ([Sales].[Customers].[CustomerId] = [Sales].[CustomerPerson].[CustomerId])
    WHERE ((CustomerName LIKE N'%' + @SearchText + N'%')
		AND ([Sales].[CustomerPerson].PersonId = @UserId))
    --ORDER BY [CustomerName]

	UNION 
	
	-- group assignment
   SELECT TOP(@MaximumRowsToReturn) [Customers].[CustomerID] as "ID", [Customers].[CustomerName] as "Name", [Customers].[IsOnCreditHold] as "IsOnCreditHold"
   FROM [Sales].[Customers]
      INNER JOIN [Sales].[CustomerCategories] ON ([Sales].[Customers].[CustomerCategoryID] = [Sales].[CustomerCategories].[CustomerCategoryID])
      INNER JOIN [Sales].[CustomerCategoryPerson] ON ([Sales].[Customers].[CustomerCategoryID] = [CustomerCategoryPerson].[CustomerCategoryID])
    WHERE ((CustomerName LIKE N'%' + @SearchText + N'%')
		AND ([Sales].[CustomerCategoryPerson].PersonId = @UserId))

END;

This approach increases the complexity and reduces the maintainability of stored procedures which have to control the reading/writing of Customer information. Several times I have extracted customer information read\write controls to a couple of database views, one for controlling read access.

CREATE VIEW [Sales].[CustomerPersonReadV1]
AS
-- Individual assignment
   SELECT [Sales].[Customers].[CustomerID], [Sales].[CustomerPerson].[PersonID], [Sales].[Customers].[CustomerCategoryID]
   FROM [Sales].[Customers]
      INNER JOIN [Sales].[CustomerPerson] ON ( [Sales].[Customers].[CustomerID] = CustomerPerson.CustomerID)

UNION -- Takes care of duplicates

-- Group assignment
   SELECT [Sales].[Customers].[CustomerID], [Sales].[CustomerCategoryPerson].[PersonID], [Sales].[Customers].[CustomerCategoryID]
   FROM [Sales].[Customers]
      --INNER JOIN [Sales].[CustomerCategories] ON ([Sales].[Customers].[CustomerCategoryID] = [Sales].[CustomerCategories].[CustomerCategoryID])
      INNER JOIN [Sales].[CustomerCategoryPerson] ON ([Sales].[Customers].[CustomerCategoryID] = [CustomerCategoryPerson].[CustomerCategoryID])

The other database for controlling write access

CREATE VIEW [Sales].[CustomerPersonWriteV1]
AS
-- Individual assignment
   SELECT [Sales].[Customers].[CustomerID], [Sales].[CustomerPerson].[PersonID], [Sales].[Customers].[CustomerCategoryID]
   FROM [Sales].[Customers]
      INNER JOIN [Sales].[CustomerPerson] ON (([Sales].[Customers].[CustomerID] = [CustomerPerson].[CustomerID]) AND ([Sales].[CustomerPerson].[IsWritable] = 1))

UNION -- Takes care of duplicates

-- Group assignment
   SELECT [Sales].[Customers].[CustomerID], [Sales].[CustomerCategoryPerson].[PersonID], [Sales].[Customers].[CustomerCategoryID]
   FROM [Sales].[Customers]
      INNER JOIN [Sales].[CustomerCategories] ON ([Sales].[Customers].[CustomerCategoryID] = [Sales].[CustomerCategories].[CustomerCategoryID])
      INNER JOIN [Sales].[CustomerCategoryPerson] ON ([Sales].[Customers].[CustomerCategoryID] = [CustomerCategoryPerson].[CustomerCategoryID] AND ([Sales].[CustomerCategoryPerson].[IsWritable] = 1))

The versioning of database views uses the same approach as stored procedures. When a view is updated (the columns returned changes , updated constraints etc.) the version number is incremented. Then we work through the dependencies list checking and updating the view version used and re-testing.

SQL Server Management Studio displaying objects which depend on the view

These two views are the UNION of the users individual and group access permissions. (If a user has Write they also have Read access). This reduces the complexity of stored procedures used for reading from and writing to the Customer table.

CREATE PROCEDURE [Sales].[CustomersListV1]
@UserId as int
AS
BEGIN
SELECT [Customers].[CustomerID] as "ID", [Customers].[CustomerName] as "Name", [Customers].[IsOnCreditHold] as "IsOnCreditHold"
	FROM [Sales].[Customers]
		INNER JOIN [Sales].[CustomerPersonReadV1] ON ([Sales].[Customers].[CustomerID] = [Sales].[CustomerPersonReadV1].CustomerID)
    WHERE ([Sales].[CustomerPersonReadV1].PersonId = @UserId)
	ORDER BY Name
END

The GET method of the Customer controller returns a list of all the Customers the current user has read only access to using their individual and group assignment.

[HttpGet(), Authorize(Roles = "SalesPerson,SalesAdministrator")]
[ProducesResponseType(StatusCodes.Status200OK, Type = typeof(List<Models.CustomerListDtoV1>))]
public async Task<ActionResult<IEnumerable<Models.CustomerListDtoV1>>> Get()
{
      IEnumerable<Models.CustomerListDtoV1> response;

      using (SqlConnection db = new SqlConnection(this.connectionString))
      {
         response = await db.QueryWithRetryAsync<Models.CustomerListDtoV1>(sql: "[Sales].[CustomersListV1]", param: new { userId = HttpContext.PersonId() }, commandType: CommandType.StoredProcedure);
      }

   return this.Ok(response);
}

The CustomerPersonWriteV1 view is used to stop users without IsWritable set (individual or group) updating a Customers IsOnCreditHold flag.

CREATE PROCEDURE [Sales].[CustomerCreditHoldStatusUpdateV1]
@UserID int,
@CustomerId int,
@IsOnCreditHold Bit
AS
BEGIN
    UPDATE [Sales].[Customers]
    SET IsOnCreditHold = @IsOnCreditHold, LastEditedBy = @UserID
	FROM [Sales].[Customers]
		INNER JOIN [Sales].[CustomerPersonWriteV1] ON ([Sales].[Customers].[CustomerID] = [Sales].[CustomerPersonWriteV1].CustomerID)
    WHERE (([Sales].[CustomerPersonWriteV1].PersonId = @UserId) 
		AND ([Sales].[Customers].[CustomerID] = @CustomerId )
		AND (IsOnCreditHold <> @IsOnCreditHold))
	
END

The PUT CreditHold method uses a combination of roles (Aministrator,SalesAdministrator,SalesPerson) and database views (CustomerPersonWriteV1) to control the updating of customer data.

[HttpPut("{customerId}/CreditStatus", Name ="CreditHold")]
[Authorize(Roles = "Aministrator,SalesAdministrator,SalesPerson")]
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status409Conflict)]
public async Task<IActionResult> CustomerCeditHold(int customerId, [FromBody] Models.CustomerCreditHoldUpdateV1 request )
{
    request.UserId = HttpContext.PersonId();
    request.CustomerId = customerId;

    using (SqlConnection db = new SqlConnection(connectionString))
    {
        if (await db.ExecuteWithRetryAsync("[Sales].[CustomerCreditHoldStatusUpdateV1]", param: request, commandType: CommandType.StoredProcedure) != 1)
        {
            logger.LogWarning("Person {UserId} Customer {CustomerId} IsOnCreditHold {IsOnCreditHold} update failed", request.UserId, request.CustomerId, request.IsOnCreditHold);

            return this.Conflict();
        }
    }

    return this.Ok();
}

My customers usually don’t have a lot of automated testing so minimising the impact of changes across the database and codebase is critical. Sometimes we duplicate code (definitely not DRY) so that the amount of functionality that has to be retested is reduced. We ensure this is time allocated for revisiting these decisions and remediating as required.

.NET Core web API + Dapper – Authorisation Permissions

The permissions required for an on-premises system running in a trusted environment are often minimalist. The World Wide Importers database People table has IsSystemUser, IsEmployee, IsSalesperson which is representative of the granularity of permissions I have encountered in Windows Forms .NET, ASP.NET Web Forms and other “legacy” applications.

CREATE TABLE [Application].[People](
    [PersonID] [int] NOT NULL,
    [FullName] nvarchar NOT NULL,
    [PreferredName] nvarchar NOT NULL,
    [SearchName] AS (concat([PreferredName],N' ',[FullName])) PERSISTED NOT NULL,
    [IsPermittedToLogon] [bit] NOT NULL,
    [LogonName] nvarchar NULL,
    [IsExternalLogonProvider] [bit] NOT NULL,
    [HashedPassword] varbinary NULL,
    [IsSystemUser] [bit] NOT NULL,
    [IsEmployee] [bit] NOT NULL,
    [IsSalesperson] [bit] NOT NULL,
    [UserPreferences] nvarchar NULL,
    [PhoneNumber] nvarchar NULL,
    [FaxNumber] nvarchar NULL,
    [EmailAddress] nvarchar NULL,
    [Photo] varbinary NULL,
    [CustomFields] nvarchar NULL,
    [OtherLanguages] AS (json_query([CustomFields],N'$.OtherLanguages')),
    [LastEditedBy] [int] NOT NULL,
    [ValidFrom] datetime2 GENERATED ALWAYS AS ROW START NOT NULL,
    [ValidTo] datetime2 GENERATED ALWAYS AS ROW END NOT NULL,
CONSTRAINT [PK_Application_People] PRIMARY KEY CLUSTERED
(
    [PersonID] ASC
)

The existing application appears to have a method for a Person to change their password which calls the [Website].[ChangePassword] stored procedure (I was surprised that the stored procedure didn’t set the LastEditedBy value).

CREATE PROCEDURE [Website].[ChangePassword]
@PersonID int,
@OldPassword nvarchar(40),
@NewPassword nvarchar(40)
WITH EXECUTE AS OWNER
AS
BEGIN
    SET NOCOUNT ON;
    SET XACT_ABORT ON;

    UPDATE [Application].People
    SET IsPermittedToLogon = 1,
        HashedPassword = HASHBYTES(N'SHA2_256', @NewPassword + FullName)
    WHERE PersonID = @PersonID
    AND PersonID <> 1
    AND HashedPassword = HASHBYTES(N'SHA2_256', @OldPassword + FullName);

    IF @@ROWCOUNT = 0
    BEGIN
        PRINT N'The PersonID must be valid, and the old password must be valid.';
        PRINT N'If the user has also changed name, please contact the IT staff to assist.';
        THROW 51000, N'Invalid Password Change', 1;
        RETURN -1;
    END;
END;

The new version removes the PersonId special case (Assumed that PersonId 1 can’t logon and the use of Throw). I think the use of Throw can add significant complexity to the exception handling of the WebAPI controller that calls the stored procedure

ALTER PROCEDURE [Website].[PersonPasswordChangeV1]
	@UserID int,
	@PasswordOld nvarchar(40),
	@PasswordNew nvarchar(40)
WITH EXECUTE AS OWNER
AS
BEGIN
    UPDATE [Application].People
    SET IsPermittedToLogon = 1
        ,HashedPassword = HASHBYTES(N'SHA2_256', @PasswordNew + FullName)
		,LastEditedBy = @UserID
    WHERE ((PersonID = @UserID )
		AND (HashedPassword = HASHBYTES(N'SHA2_256', @PasswordOld + FullName)))
END;

The PasswordChange method of the Person Controller only requires the caller to be authenticated.

/// <summary>
/// Changes current user's password.
/// </summary>
/// <param name="request">Current password and new password</param>
/// <response code="200">Password changed.</response>
/// <response code="401">Unauthorised, bearer token missing or expired.</response>
/// <response code="409">Previous password invalid or User name has changed.</response>
[Authorize()]
[HttpPut(Name = "PasswordChange")]
public async Task<ActionResult> PasswordChange([FromBody] Models.PersonPasswordChangeRequest request)
{
    request.UserID = HttpContext.PersonId();

    using (SqlConnection db = new SqlConnection(connectionString))
    {
        if (await db.ExecuteWithRetryAsync("[WebSite].[PersonPasswordChangeV1]", param: request, commandType: CommandType.StoredProcedure) != 1)
        {
            logger.LogWarning("Person {0} password change failed", request.UserID);

            return this.Conflict();
        }
    }

    return this.Ok();
}

The new application will have functionality for resetting a Person’s password. Access to this functionality will be restricted to people with the “Administrator” and “PasswordReset” roles.

CREATE PROCEDURE [Website].[PersonPasswordResetV1]
@UserID int,
@PersonID int,
@Password nvarchar(40)
WITH EXECUTE AS OWNER
AS
BEGIN
    UPDATE [Application].People
    SET IsPermittedToLogon = 1
        ,HashedPassword = HASHBYTES(N'SHA2_256', @Password + FullName)
		,LastEditedBy = @UserID
    WHERE PersonID = @PersonID
END;

One of the conventions we often use, is that the first parameter of any stored procedure that is called once a User has logged on is their unique identifier which is used for data access permissions and change tracking.

[Authorize(Roles = "Administrator")]
[HttpPut("{personId:int}", Name = "PasswordReset")]
public async Task<ActionResult> PasswordReset([Range(1, int.MaxValue, ErrorMessage = "Person id must greater than 1")] int personId, [FromBody] Models.PersonPasswordResetRequest request)
{
    request.UserId = HttpContext.PersonId();
    request.PersonID = personId;

    using (SqlConnection db = new SqlConnection(connectionString))
    {
        if (await db.ExecuteWithRetryAsync("[WebSite].[PersonPasswordResetV1]", param: request, commandType: CommandType.StoredProcedure) != 1)
        {
            logger.LogWarning("Person {0} password change failed", request.PersonID);

            return this.Conflict();
        }
    }

    return this.Ok();
}

For a couple of applications, we have added “Permissions” and “PersonPermissions” tables alongside the existing authorisation functionality to reduce the likely hood of any unintended side effects.

CREATE TABLE [Application].[Permissions](
	[PermissionID] [int] IDENTITY(1,1) NOT NULL,
	[Name] [nvarchar](20) NOT NULL,
	[Description] [nvarchar](50) NOT NULL,
	[LastEditedBy] [int] NOT NULL,
	[ValidFrom] [datetime2](7) GENERATED ALWAYS AS ROW START HIDDEN NOT NULL,
	[ValidUntil] [datetime2](7) GENERATED ALWAYS AS ROW END HIDDEN NOT NULL,
 CONSTRAINT [PK_Permissions] PRIMARY KEY CLUSTERED 
(
	[PermissionID] ASC
)...

We try and keep the names of the permissions short, so the token doesn’t get too large.

CREATE TABLE [Application].[PersonPermissions](
	[PersonPermissionId] [int] IDENTITY(1,1) NOT NULL,
	[PersonId] [int] NOT NULL,
	[PermIssionId] [int] NOT NULL,
	[Active] [bit] NOT NULL,
	[ValidFrom] [datetime2](7) GENERATED ALWAYS AS ROW START HIDDEN NOT NULL,
	[ValidUntil] [datetime2](7) GENERATED ALWAYS AS ROW END HIDDEN NOT NULL,
 CONSTRAINT [PK_PersonPermissions] PRIMARY KEY CLUSTERED 
(
	[PersonPermissionId] ASC
)...
Permissions, PersonPermissions and People with Foreign Keys

The additional permissions (from the Person record) and the PersonPermissions table required some modifications to the PersonAuthenticateLookupByLogonNameV1 stored procedure and the addition of the PersonPermissionsByPersonIdV1 stored procedure.

ALTER PROCEDURE [Website].[PersonAuthenticateLookupByLogonNameV2]
@LogonName nvarchar(50),
@Password nvarchar(40)
AS
BEGIN
	SELECT PersonID
		,FullName
		,EmailAddress
		,IsSystemUser
		,IsEmployee
		,IsSalesPerson
	FROM [Application].[People]
	WHERE (( LogonName = @LogonName)
		AND (IsPermittedToLogon = 1)
		AND (HASHBYTES(N'SHA2_256', @Password + FullName) = HashedPassword))

The IsSystemUser, IsEmployee and IsSalesPerson bit flags were added to the stored procedure and Data Transfer Object(DTO)

private class PersonAuthenticateLogonDetailsDto
{
    public int PersonID { get; set; }    

    public string FullName { get; set; }

    public string EmailAddress { get; set; }

    public bool IsSystemUser { get; set; }

    public bool IsEmployee { get; set; }

    public bool IsSalesPerson { get; set; }
}

The PersonPermissionsByPersonIdV1 retrieves a list of the permissions of the User who has been authenticated.

ALTER PROCEDURE [Website].[PersonPermissionsByPersonIdV1]
	@PersonId AS int
AS
BEGIN

	SELECT [Application].[Permissions].[Name]
	FROM [Application].[Permissions]
		INNER JOIN [Application].[PersonPermissions] ON ([Application].[Permissions].PermissionID = [Application].[PersonPermissions].[PermissionId] )
	WHERE [Application].[PersonPermissions].[PersonId] = @PersonId
	ORDER BY [Application].[Permissions].[Name]

END

The Person’s permissions(effectively roles) are added as claims, the IsSystemUser, IsEmployee and IsSalesPerson flags are also added to the list of claims so they can be used in the new application.

[HttpPost("logon")]
public async Task<ActionResult> Logon([FromBody] Models.LogonRequest request )
{
    PersonAuthenticateLogonDetailsDto userLogonUserDetails;
    IEnumerable<string> permissions;
    var claims = new List<Claim>();

    using (SqlConnection db = new SqlConnection(configuration.GetConnectionString("WorldWideImportersDatabase")))
    {
        userLogonUserDetails = await db.QuerySingleOrDefaultWithRetryAsync<PersonAuthenticateLogonDetailsDto>("[Website].[PersonAuthenticateLookupByLogonNameV2]", param: request, commandType: CommandType.StoredProcedure);
        if (userLogonUserDetails == null)
        {
            logger.LogWarning("Login attempt by user {0} failed", request.LogonName);

           return this.Unauthorized();
        }

        // Lookup the Person's permissions
        permissions = await db.QueryWithRetryAsync<string>("[Website].[PersonPermissionsByPersonIdV1]", new { userLogonUserDetails.PersonID }, commandType: CommandType.StoredProcedure);
    }

    // Setup the primary SID + name info
    claims.Add(new Claim(ClaimTypes.PrimarySid, userLogonUserDetails.PersonID.ToString()));
    if (userLogonUserDetails.IsSystemUser)
    {
       claims.Add(new Claim(ClaimTypes.Role, "SystemUser"));
    }
    if (userLogonUserDetails.IsEmployee)
    {
       claims.Add(new Claim(ClaimTypes.Role, "Employee"));
    }
    if (userLogonUserDetails.IsSalesPerson)
    {
        claims.Add(new Claim(ClaimTypes.Role, "SalesPerson"));
    }

    foreach(string permission in permissions)
    {
        claims.Add(new Claim(ClaimTypes.Role, permission));
    }

    var authSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(jwtIssuerOptions.SecretKey));

    var token = new JwtSecurityToken(
         issuer: jwtIssuerOptions.Issuer,
          audience: jwtIssuerOptions.Audience, 
          expires: DateTime.UtcNow.Add(jwtIssuerOptions.TokenExpiresAfter),
          claims: claims,
          signingCredentials: new SigningCredentials(authSigningKey, SecurityAlgorithms.HmacSha256));
                
    return this.Ok(new
   {
        token = new JwtSecurityTokenHandler().WriteToken(token),
        expiration = token.ValidTo,
    });
}

We try to reduce the number of roles a User requires by having core roles (Administrator, Sales consultant, Warehouse administrator etc.) with additional roles for each task that can be added as required (ResetPassword, CustomerIsOnCreditHold Set/Clear etc.)

The Get Method of Authorisation controller returns a list of the User’s Roles which can be used to enable/disable functionality of the user interface.

/// <summary>
/// Gets a list of the current User's roles.
/// </summary>
/// <response code="200">List of claims returned.</response>
/// <response code="401">Unauthorised, bearer token missing or expired.</response>
/// <returns>list of claims.</returns>
[HttpGet]
[ProducesResponseType(StatusCodes.Status200OK, Type = typeof(List<string>))]
public List<string> Get()
{
    List<string> claimNames = new List<string>();

    foreach (var claim in this.User.Claims.Where(c => c.Type == ClaimTypes.Role))
    {
        claimNames.Add(claim.Value);
    }

    return claimNames;
}

We have found this approach to be a robust way to add granular authorisation for new functionality to a “legacy’ application without breaking the existing solution. I have ignored a user being disabled after a number of failed logins, password complexity rules etc. as these tend to be application specific and not really related to the use of Dapper.

Most blog posts talk about building “green fields” applications, I have found hardly any cover “muddy fields” development where you have to deal with “legacy” code.

Not all “legacy” code is bad, I work on one code base which is nearly 20years old. It started as a spreadsheet plug-in and has grown of time to a SaaS application. There is very little of the original code left it has just been carefully re-factored over the years with time allocated to chip away at technical debt.