Windows 10 IoT Core LoRa library

I have a pair of Windows 10 IoT Core nRF24L01 field gateway projects, one for AdaFruit.IO and the other for Azure IoTHub (Including Azure IoT Central). I use these field gateways for small scale indoor and outdoor deployments.

For larger systems e.g a school campus I was looking for something with a bit more range (line of site + in building penetration) and clients with lower power consumption (suitable for long term battery or solar power).

Other makers had had success with RFM69(proprietary) and RFM9X (LoRA) based devices and shields/hats so I had a look at both technologies.

To kick things off I purchased

I then did some searching and downloaded two commonly used libraries

Initially I trialled the emmellsoft Windows 10 IoT Core Dragino.LoRa code on a couple of Raspberry PI devices.

RPIDraginoP2P

After updating the Windows 10 Min/Max versions, plus the NuGet packages and setting the processor type to ARM the code compiled, downloaded and ran which was a pretty good start.

I could see messages being transmitted and received by the two devices


Packet RSSI: -33, RSSI: -91, SNR: 8, Length: 5

Message Received: CRC OK, Rssi=-91, PacketRssi=-33, PacketSnr=8, Buffer:[55, ff, 00, aa, 01], 2018-07-30 09:27:48

Successfully sent in 110 milliseconds.

Packet RSSI: -15, RSSI: -100, SNR: 9.2, Length: 5

Message Received: CRC OK, Rssi=-100, PacketRssi=-15, PacketSnr=9.2, Buffer:[55, ff, 00, aa, 02], 2018-07-30 09:27:53

Successfully sent in 36 milliseconds.

Packet RSSI: -35, RSSI: -101, SNR: 9, Length: 5

Message Received: CRC OK, Rssi=-101, PacketRssi=-35, PacketSnr=9, Buffer:[55, ff, 00, aa, 03], 2018-07-30 09:27:58

Successfully sent in 36 milliseconds.

I added my first attempt at device configuration for New Zealand (based on EU settings) in Dragino.LoRa\Radio\TransceiverSettings.cs


public static readonly TransceiverSettings ANZ915 = new TransceiverSettings(

RadioModemKind.Lora,

915000000,

BandWidth.BandWidth_125_00_kHz,

SpreadingFactor.SF7,

CodingRate.FourOfFive,

8,

true,

false,

LoraSyncWord.Public);

The LoraSyncWord.Public would turn out to be a problem later!

Then I modified the sender and receiver sample application MainPage.xaml.cs files to reference my settings


private static TransceiverSettings GetRadioSettings()

{

// *********************************************************************************************

// #1/2. YOUR EDITING IS REQUIRED HERE!

//

// Choose transeiver settings:

// *********************************************************************************************

return TransceiverSettings.Standard.ANZ915;

}

I modified one of the RadioHead sample Arduino applications (centre frequency) and deployed it to a LoRa MiniDev device. I could see messages getting sent but they were not getting received by the RPI(s).

So I dumped the registers for the SX127X device in the HopeRF RFM95 module on both devices,

DraginoLoraMinDev

From the device on RPI Hat


SX1276/77/78/79 detected, starting.

1-85

2-1A

3-B

4-0

5-52

6-E4

7-C0

8-0

9-85

A-9

B-2B

C-23

D-0

E-80

F-0

10-0

11-0

12-0

13-0

14-0

15-0

16-0

17-0

18-4

19-0

1A-0

1B-42

1C-0

1D-72

1E-74

1F-9F

20-0

21-8

22-1

23-FF

24-0

25-0

26-4

27-0

28-0

29-0

2A-0

2B-0

2C-9

2D-50

2E-14

2F-45

30-55

31-C3

32-5

33-27

34-1C

35-A

36-3

37-A

38-42

39-34

The LoRa transceiver is initiated successfully.

I printed out the Radiohead and emmellsoft registers then manually compared them using the SX1276 datasheet for reference.

I found the 3 registers which contain the MSB, ISB and LSB for the centre frequency weren’t being calculated correctly (checked this by changing the frequency to 434MHz and comparing the register values to the worked example in the datasheet).

I manually “brute forced” the centre frequency registers in LoRaTransceiver.cs Init() and the RPI could then detect a signal but couldn’t decode the messages.

I went back to the Register dumps and found the SyncWord (odd name as it is a byte) were different. After updating the RPI settings the devices could exchange packets..


const double RH_RF95_FXOSC = 32000000.0;

const double RH_RF95_FSTEP = RH_RF95_FXOSC / 524288.0;

long frf = (long)(Settings.Frequency / RH_RF95_FSTEP);

byte[] bytes = BitConverter.GetBytes(frf);

byte[] x6 = { bytes[2] };

RegisterManager.WriteRegister(6, x6);

byte[] x7 = { bytes[1] };

RegisterManager.WriteRegister(7, x7);

byte[] x8 = { bytes[0] };

RegisterManager.WriteRegister(8, x8);

RegisterManager.Write(new LoraRegisterSyncWord(Settings.LoraSyncWord.Value));

This was not a long term solution, lots of code, and register setting changes with limited explanation…

ASP MVC Core V2.1 and Cross-Origin Resource Sharing

I’m working on an project for a customer which implements a number of application programming Interfaces(API) for a Single Page Application(SPA) and other clients. We are using entity tags (ETags) for versioning and the front end developers found the couldn’t access them from javascipt running in mainstream browser clients (June 2018).

The problems was understanding how Cross-Origin Resource Sharing (CORS) worked and how it interacted with our security model (API key and OAuth2.0 depending on the client)

In our scenario we first found the pre-flight check wasn’t working because in the HyperText Transfer Protocol (HTTP) OPTIONS method our X-API-KEY check was failing

OPTIONS http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
Access-Control-Request-Headers: x-api-key
Access-Control-Request-Method: GET
Accept-Encoding: gzip, deflate
Content-Length: 0
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Server: Kestrel
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:48:30 GMT

13
API Key is invalid.
0

So I disabled X-API-KEY validation in startup.cs

public async Task Invoke(HttpContext context)
{
   if (context.Request.Method == "OPTIONS")
   {
      await this.next.Invoke(context);
      return;
   }

   var claims = new List();
…

OPTIONS then worked

OPTIONS http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
Access-Control-Request-Headers: x-api-key
Access-Control-Request-Method: GET
Accept-Encoding: gzip, deflate
Content-Length: 0
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 404 Not Found
Server: Kestrel
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:52:20 GMT
Content-Length: 0

I then turned on CORS allowing pretty much anything

public void ConfigureServices(IServiceCollection services)
{
   services.AddCors(options =>
   {
      options.AddPolicy("CorsPolicy",
      builder => builder.AllowAnyOrigin()
         .AllowAnyMethod()
         .AllowAnyHeader()
         .AllowCredentials());
   });
   services.AddMvc();
}

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
   if (env.IsDevelopment())
   {
      app.UseDeveloperExceptionPage();
   }

   TelemetryConfiguration.Active.InstrumentationKey = this.configuration.GetSection("ApplicationInsights").GetSection("InstrumentationKey").Value;

   loggerFactory.AddLog4Net();
   this.log.Info("Startup.Configure called");

   app.ApplyUserKeyValidation();
   app.UseCors("CorsPolicy");
   app.UseMvc();
   }
}

OPTIONS then worked

OPTIONS http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
Access-Control-Request-Headers: x-api-key
Access-Control-Request-Method: GET
Accept-Encoding: gzip, deflate
Content-Length: 0
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 204 No Content
Vary: Origin
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: x-api-key
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:57:33 GMT

GET then worked

GET http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
X-API-KEY: ABCDEFGHIJKLMNOPQRSTUVWXYZ
Accept-Language: en-NZ
Accept-Encoding: gzip, deflate
If-None-Match: 00-00-00-00-00-00-00-76
Host: xyz.azurewebsites.net
Connection: Keep-Alive

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
ETag: 00-00-00-00-00-00-00-76
Vary: Origin,Accept-Encoding
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:57:34 GMT
Content-Length: 2216

[{"...."}}

But HEAD didn’t work

HEAD http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
X-API-KEY: ABCDEFGHIJKLMNOPQRSTUVWXYZ
Accept-Language: en-NZ
Accept-Encoding: gzip, deflate
If-None-Match: 00-00-00-00-00-00-00-76
Host: xyz.azurewebsites.net
Connection: Keep-Alive

HTTP/1.1 400 Bad Request
Content-Length: 0
Vary: Origin
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:59:55 GMT

From the Application Insights logging and RestTest client (which I ran locally and remotely) I could see that the client side code couldn’t access the value of our eTag.  It had to be “exposed”

public void ConfigureServices(IServiceCollection services)
{
   services.AddCors(options =>
   {
      options.AddPolicy("CorsPolicy",
            builder => builder.AllowAnyOrigin()
            .AllowAnyMethod()
            .AllowAnyHeader()
            .WithExposedHeaders("etag")
            .AllowCredentials()
         );
      });
      services.AddMvc();
   }
...

GET then worked

GET http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
X-API-KEY: ABCDEFGHIJKLMNOPQRSTUVWXYZ
ETag: 00-00-00-00-00-00-00-76
Accept-Language: en-NZ
Accept-Encoding: gzip, deflate
If-None-Match: 00-00-00-00-00-00-00-76
Host: xyz.azurewebsites.net
Connection: Keep-Alive

HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
ETag: 00-00-00-00-00-00-00-76
Vary: Origin,Accept-Encoding
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 07:53:41 GMT

[{"...."}}

HEAD then worked

OPTIONS http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
Access-Control-Request-Headers: x-api-key,etag
Access-Control-Request-Method: HEAD
Accept-Encoding: gzip, deflate
Content-Length: 0
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 204 No Content
Vary: Origin
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: x-api-key,etag
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 07:57:31 GMT

HEAD http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
X-API-KEY: ABCDEFGHIJKLMNOPQRSTUVWXYZ
ETag: 00-00-00-00-00-00-00-76
Accept-Language: en-NZ
Accept-Encoding: gzip, deflate
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 304 Not Modified
Vary: Origin
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 07:57:31 GMT

I had some oddness with releasing code updates which I think was down to caching of pre-flight request responses.
Next steps tidy up the headers etc. and lock the CORS configuration down to expose the minimum necessary required for the application to work.

Azure Meetup-Budget tank of 91 IoT

The premise of my Azure Meetup presentation was could you build an interesting project on a rainy weekend afternoon with a constrained budget (tank of 91 octane petrol) and minimal soldering .

Budget

Our family car is a VW Passat V6 4Motion which has a 62 Litre tank. The driver usually doesn’t usually stop to fill up until the fuel light has been on for a bit which helped.

PetrolReceipt

Based on the most recent receipt the budget was NZD132.

Where possible I purchased parts locally (the tech equivalent of food miles) or on special.

My bill of materials (prices as at 2018-06) was on budget.

The devDuino V2.2 and nRF24L01 module were USD26.20 approx. NZD37.50 (including freight) from elecrow.

Tradeoffs

I powered my Raspberry PI with a spare cellphone charger (make sure it can supply enough current to reliably power the device).

The devDuino V2.has an ATSHA204A which provides a guaranteed unique 72-bit serial number (makes it harder to screw up provisioning devices in the field).

I use a 32G MicroSD rather than a 16G MicroSD card as I have had issued with 16G cards getting corrupted by more recent upgrades (possibly running out of space?)

The Raspberry PI shield requires a simple modification to enable interrupt driven operation.

My sample devDuino V2.2 client uses an external temperature and humidity sensor, modifying this code to use the onboard temperature sensor an MCP9700 will be covered in another post.

The devDuino V2 is a little bit cheaper USD15.99 NZD37.31, has the same onboard temperature sensor as the V2.2 but no unique serial number chip.

The devDuino V4.0 has an onboard HTU21D temperature + humidity sensor but no unique serial number and the batteries are expensive.

The code and deployment instructions for the nRF24L01 field gateway applications for AdaFruit.IO and Azure IoT Hub/Azure IoT Central are available on hackster.IO.

RPiWithnRF24Plate

AdaFruit.IO has free and USD10.00/month options which work well for many hobbyist projects.

AdaFruitIO

Azure Meetup Christchurch notes

For the people who came to my Azure meetup session this evening

Sources of sensors and development boards

http://www.adafruit.com
http://www.elecrow.com (watering kits)
http://www.ingenuitymicro.com (NZ based dev boards)
http://www.netduino.com (.NetMF development boards)
http://www.makerfabs.com
http://www.seeedstudio.com
http://www.tindie.com

nRF24Shields for RPI devices
http://www.tindie.com/products/ceech/new-raspberry-pi-to-nrf24l01-shield/

nRF24Shields for *duino devices in AU
embeddedcoolness.com

Raspberry PI Source in CHC
http://www.wavetech.co.nz

RFM69 & LoRa Modules
http://www.wisen.com.au

local sensor and device resellers quick turnaround
http://www.mindkits.co.nz
http://www.nicegear.co.nz

http://www.diyelectricskateboard.com

The watch development platform
http://www.hexiwear.com

http://www.gowifi.co.nz (Antennas & other wireless kit based in Rangiora)

my projects
http://www.hackster.io/KiwiBryn
io.adafruit.com/BrynHLewis/dashboards/home-environment

NLog and Application Insights

Another of my clients has an application which uses NLog and sooner or later they are going to want to move their logging to Azure Application Insights.

The application consists of a number of Azure websites and some embedded clients. The Azure applications log information to the local file system on each box but the number of boxes is growing so finding and tracing issues is becoming painful.

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights.Extensibility;
using NLog;

namespace devMobile.Azure.ApplicationInsightsNLogClient
{
   class Program
   {
      private static Logger log = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType.ToString());

      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         log.Trace("This is nLog");
         log.Debug("This is a Debug message");
         log.Info("This is a Info message");
         log.Warn("This is a Warning message");
         log.Error("This is an Error message");
         log.Fatal("This is a Fatal message");

         new Microsoft.ApplicationInsights.TelemetryClient().Flush();
      }
   }
}

Sample code ApplicationInsightsNLogClient

Two clients to go, one which uses serilog the other has a DIY system which I’m ignoring as long as possible.

Apache Log4net and Application Insights

One of my clients had built a Fintech application which uses Apache log4net and I have just finished moving the logging to Azure Application Insights.

The application used to run on a couple of dedicated servers but its core processing engine is now run on an Azure Virtual Machine Scale Set(VMSS).

The application used to log information to the local file system on each server but with more machines (upto a dozen) and them being started up and shutdown in response to customer demand this wasn’t a viable approach.

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using log4net;
using Microsoft.ApplicationInsights.Extensibility;

namespace devMobile.Azure.ApplicationInsightsLog4NetClient
{
   class Program
   {
      public static ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         log.Info("This is Log4net");
         log.Debug("This is a Debug message");
         log.Info("This is a Info message");
         log.Warn("This is a Warning message");
         log.Error("This is an Error message");
         log.Fatal("This is a Fatal message");

         new Microsoft.ApplicationInsights.TelemetryClient().Flush();
      }
   }
}

Sample code ApplicationInsightsLog4NetClient

Microsoft Enterprise Library and Application Insights

One of my clients has a largish application (120+ projects) which uses the Microsoft Patterns and Practices Enterprise Library V6 data access, exception handling, logging and transient fault handling blocks.

To get consistent logging across Classic Cloud services, Azure websites and Azure functions etc. we are in the process of moving all our diagnostics to Azure application insights.

My proof of concept uses a community developed Enterprise Library listener and it appears to be working well.

Beware the Visual Studio configuration tool plug-in rewrites and application config file removing the application insights enterprise library trace listener setup.

The code for a smallest example application is below (I pass the instrumentation key as a command line parameter).

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.Practices.EnterpriseLibrary.Logging;
using Microsoft.Practices.EnterpriseLibrary.ExceptionHandling;
namespace ApplicationInsightsEnterpriseLibraryClient
{
   class Program
   {
      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         LogWriterFactory logWriterFactory = new LogWriterFactory();
         LogWriter logWriter = logWriterFactory.Create();
         Logger.SetLogWriter(logWriter);

         ExceptionManager exceptionManager = new ExceptionPolicyFactory().CreateManager();
         ExceptionPolicy.SetExceptionManager(exceptionManager);

         logWriter.Write("This is Entlib", "General");

         logWriter.Write("Application startup", "Startup");

         logWriter.Write("General category", "General");
         logWriter.Write(new LogEntry() { Severity = System.Diagnostics.TraceEventType.Error, Categories = { "General" }, Message = "General category more complex overload", Title = "Dumpster fire" });

         try
         {
            throw new ApplicationException("Something bad has happened");
         }
         catch (Exception ex)
         {
            bool rethrow = ExceptionPolicy.HandleException(ex, "ProgramMain");
            if (rethrow)
               throw;
         }

         logWriter.Write("Application shutdown", "Shutdown");

         new Microsoft.ApplicationInsights.TelemetryClient().Flush();
      }
   }
}

Sample project ApplicationInsightsEnterpriseLibraryClient

Thanks to bveerendrakumar for sharing your code

“Don’t forget to flush” Application Insights

Revisited March 2020

An Azure solution I was working on had a .Net console application which ran on a server at the customer’s premises. It was scheduled task that uploaded some files to azure blob storage every 5 minutes.

To help with debugging I added support for Azure application Insights but after monitoring the application for a while I noticed some shutdown events were not getting uploaded.

Initially I was a bit confused because when I ran the application on my desktop it worked fine (It works on my machine). I found this was because when launched from the debugger the application would upload any files it found then wait until I pressed to exit and this was enough time for the shutdown messages to get uploaded.

The code for a smallest example application is below (I pass the instrumentation key as a command line parameter).

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;

namespace devMobile.Azure.ApplicationInsightsClientConsole
{
   class Program
   {
      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         TelemetryClient telemetryClient = new TelemetryClient();

         telemetryClient.TrackTrace("This is Application Insights native");

         telemetryClient.TrackTrace("Application startup");

         // application does stuff

         telemetryClient.TrackTrace("Application shutdown");

         telemetryClient.Flush();
      }
   }
}

Sample project AzureApplicationInsightsClientConsole

Wireless field gateway Netduino client V2

This revised client is a Netduino V2Plus/V3 Ethernet/V3 Wifi device with a Silicon Labs SI7005 temperature & humidity sensor. These devices when used as sensor nodes can be battery powered and I use the Mac Address as the unique device identifier.

In this version of the protocol the message type & device identifier are nibbles packed into the first bye of the message. This saved a byte but limits the number of message types and device identifier length

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using System.Net;
using System.Text;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using Microsoft.SPOT.Net.NetworkInformation;
using devMobile.NetMF.Sensor;
using Gralin.NETMF.Nordic;
using SecretLabs.NETMF.Hardware.Netduino;

namespace devMobile.IoT.FIeldGateway.Netduino.Client
{
   class Client
   {
      private const byte nRF24Channel = 10;
      private const NRFDataRate nRF24DataRate = NRFDataRate.DR250kbps;
      private readonly byte[] nRF24ClientAddress = Encoding.UTF8.GetBytes("T&H01");
      private readonly byte[] nRF24BaseStationAddress = Encoding.UTF8.GetBytes("Base1");
      private static byte[] deviceIdentifier;
      private readonly OutputPort led = new OutputPort(Pins.ONBOARD_LED, false);
      private readonly NRF24L01Plus radio;
      private readonly SiliconLabsSI7005 sensor = new SiliconLabsSI7005();

      public Client()
      {
         radio = new NRF24L01Plus();
      }

      public void Run()
      {
         // Configure the nRF24 hardware
         radio.OnDataReceived += OnReceive;
         radio.OnTransmitFailed += OnSendFailure;
         radio.OnTransmitSuccess += OnSendSuccess;

         radio.Initialize(SPI.SPI_module.SPI1, Pins.GPIO_PIN_D7, Pins.GPIO_PIN_D3, Pins.GPIO_PIN_D2);
         radio.Configure(nRF24ClientAddress, nRF24Channel, nRF24DataRate);
         radio.Enable();

         // Setup the device unique identifer, in this case the hardware MacAddress
         deviceIdentifier = NetworkInterface.GetAllNetworkInterfaces()[0].PhysicalAddress;
         Debug.Print(" Device Identifier : " + BytesToHexString(deviceIdentifier));

         Timer humidityAndtemperatureUpdates = new Timer(HumidityAndTemperatureTimerProc, null, 15000, 15000);

         Thread.Sleep(Timeout.Infinite);
      }

          private void HumidityAndTemperatureTimerProc(object state)
      {
         led.Write(true);

         double humidity = sensor.Humidity();
         double temperature = sensor.Temperature();

         Debug.Print("H:" + humidity.ToString("F1") + " T:" + temperature.ToString("F1"));
         string values = "T " + temperature.ToString("F1") + ",H " + humidity.ToString("F0");

         // Stuff the single byte header ( payload type nibble & deviceIdentifierLength nibble ) + deviceIdentifier into first byte of payload
         byte[] payload = new byte[ 1 + deviceIdentifier.Length + values.Length];
         payload[0] =  (byte)((1 <> 4];

            // Mask off the upper 4 bits to get the rest of it.
            hexString += hexChars[bytes[b] & 0x0F];
         }

         return hexString;
      }
   }
}

Bill of materials (prices as at March 2018)

.Net version of SQL Server PWDCompare

One of my customers Hedgebook has a Microsoft SQL Server database with passwords that have been secured using PWDENCRYPT and PWDCOMPARE As part of a migration plan (away from this approach) we need to be able to validate passwords against hashes that have been generated with many versions of Microsoft SQL Server.

After some searching I found a stackoverflow post which described how to validate hashes up to SQL Server 2012 and I have added code to support more modern versions of SQL Server.

I had a chat with my boss and he approved me posting a console application wrapper for an anonymised version of the code as an aide to other developers.

This sample code is not production ready it is just to illustrate how the password hashes for older and newer versions of SQL Server can be validated in C#

//---------------------------------------------------------------------------------
// Copyright ® Feb 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// based on https://stackoverflow.com/questions/43879003/how-can-i-manually-check-passwords-hashed-with-sql-server-pwdencrypt-in-c-sharp
//
// Have added implementation for more modern SQL Server boxes and built as a console application
//---------------------------------------------------------------------------------
namespace devMobile.SqlServer.PWDCompareDemo
{
   using System;
   using System.Data.SqlClient;
   using System.Linq;
   using System.Security.Cryptography;
   using System.Text;

   public class Program
   {
      private const int DatabasePasswordHashLength = 256;
      private const int HeaderLength = 2;
      private const int SaltLength = 4;
      private const int Sha1HashLength = 20;
      private const int Sha512HashLength = 64;

      public static void Main(string[] args)
      {
         if (args.Length != 3)
         {
            Console.WriteLine("Expecting ConnectionString UserID Password");
            Console.WriteLine("Press ");
            Console.ReadLine();
            return;
         }

         string connectionString = args[0];
         string userId = args[1];
         string password = args[2];

         using (SqlConnection conn = new SqlConnection(connectionString))
         {
            conn.Open();

            using (SqlCommand cmd = new SqlCommand("SELECT Password FROM Users WHERE UserID=@UserID", conn))
            {
               cmd.Parameters.AddWithValue("@UserID", userId);

               using (SqlDataReader reader = cmd.ExecuteReader())
               {
                  if (reader.Read())
                  {
                     byte[] databasePasswordHash = new byte[DatabasePasswordHashLength];
                     reader.GetBytes(0, 0, databasePasswordHash, 0, databasePasswordHash.Length);

                     int header = BitConverter.ToChar(databasePasswordHash, 0);
                     byte[] salt = new byte[SaltLength];
                     Buffer.BlockCopy(databasePasswordHash, HeaderLength, salt, 0, salt.Length);

                     switch (header)
                     {
                        case 1: //SHA1 encryption for old SQL Server
                           byte[] sha1Hash = new byte[Sha1HashLength];
                           Buffer.BlockCopy(databasePasswordHash, HeaderLength + SaltLength, sha1Hash, 0, sha1Hash.Length);

                           HashAlgorithm sha1Hasher = SHA1.Create();
                           byte[] sha1Result = sha1Hasher.ComputeHash(Encoding.Unicode.GetBytes(password + Encoding.Unicode.GetString(salt)));
                           if (sha1Hash.SequenceEqual(sha1Result))
                           {
                              Console.WriteLine("SHA1 Password is good");
                           }
                           else
                           {
                              Console.WriteLine("SHA1 Password is bad");
                           }

                           break;

                        case 2: //SHA2-512 encryption for modern SQL Server
                           byte[] sha512Hash = new byte[Sha512HashLength];
                           Buffer.BlockCopy(databasePasswordHash, HeaderLength + SaltLength, sha512Hash, 0, sha512Hash.Length);

                           HashAlgorithm sha512Hasher = SHA512.Create();
                           byte[] sha512Result = sha512Hasher.ComputeHash(Encoding.Unicode.GetBytes(password + Encoding.Unicode.GetString(salt)));
                           if (sha512Hash.SequenceEqual(sha512Result))
                           {
                              Console.WriteLine("SHA512 Password is good");
                           }
                           else
                           {
                              Console.WriteLine("SHA512 Password is bad");
                           }

                           break;

                        default:
                           Console.WriteLine("Unknown header value something bad has happened");
                           break;
                     }
                  }
               }
            }
         }

         Console.WriteLine("Press ");
         Console.ReadLine();
      }
   }
}