Cayenne Low Power Payload (LPP) Encoder

Reducing the size of message payloads is important for LoRa/LoRaWAN communications, as it reduces power consumption and bandwidth usage. One of the more common formats is myDevices Cayenne Low Power Payload(LPP) which is based on the IPSO Alliance Smart Objects Guidelines and is natively supported by The Things Network(TTN).

 private enum DataType : byte
{
   DigitalInput = 0, // 1 byte
   DigitialOutput = 1, // 1 byte
   AnalogInput = 2, // 2 bytes, 0.01 signed
   AnalogOutput = 3, // 2 bytes, 0.01 signed
   Luminosity = 101, // 2 bytes, 1 lux unsigned
   Presence = 102, // 1 byte, 1
   Temperature = 103, // 2 bytes, 0.1°C signed
   RelativeHumidity = 104, // 1 byte, 0.5% unsigned
   Accelerometer = 113, // 2 bytes per axis, 0.001G
   BarometricPressure = 115, // 2 bytes 0.1 hPa Unsigned
   Gyrometer = 134, // 2 bytes per axis, 0.01 °/s
   Gps = 136, // 3 byte lon/lat 0.0001 °, 3 bytes alt 0.01m
}

My implementation was “inspired” by the myDevices C/C++ sample code. The first step was to allocate a buffer to store the byte encoded values. I pre allocated the buffer to try and reduce the impacts of garbage collection. The code uses a manually incremented index into the buffer for performance reasons, plus the inconsistent support of System.Collections.Generic and Language Integrated Query(LINQ) on my three embedded platforms. The maximum length message that can be sent is limited by coding rate, duty cycle and bandwidth of the LoRa channel.

public Encoder(byte bufferSize)
{
   if ((bufferSize < BufferSizeMinimum) || ( bufferSize > BufferSizeMaximum))
   {
      throw new ArgumentException($"BufferSize must be between {BufferSizeMinimum} and {BufferSizeMaximum}", "bufferSize");
   }

   buffer = new byte[bufferSize];
}

For a simple data types like a digital input a single byte (True or False ) is used. The channel parameter is included so that multiple values of the same data type can be included in a message.

public void DigitalInputAdd(byte channel, bool value)
{
   if ((index + DigitalInputSize) > buffer.Length)
   {
     throw new ApplicationException("DigitalInputAdd insufficent buffer capacity");
   }

   buffer[index++] = channel;
   buffer[index++] = (byte)DataType.DigitalInput;
   // I know this is fugly but it works on all platforms
   if (value)
   {
      buffer[index++] = 1;
   }
   else
   {
      buffer[index++] = 0;
   }
}

For more complex data types like a Global Positioning System(GPS) location (Latitude, Longitude and Altitude) the values are converted to 32bit signed integers and only 3 of the 4 bytes are used.

public void GpsAdd(byte channel, float latitude, float longitude, float meters)
{
   if ((index + GpsSize) > buffer.Length)
   {
     throw new ApplicationException("GpsAdd insufficent buffer capacity");
   }

   int lat = (int)(latitude * 10000);
   int lon = (int)(longitude * 10000);
   int alt = (int)(meters * 100);

   buffer[index++] = channel;
   buffer[index++] = (byte)DataType.Gps;

   buffer[index++] = (byte)(lat >> 16);
   buffer[index++] = (byte)(lat >> 8);
   buffer[index++] = (byte)lat;
   buffer[index++] = (byte)(lon >> 16);
   buffer[index++] = (byte)(lon >> 8);
   buffer[index++] = (byte)lon;
   buffer[index++] = (byte)(alt >> 16);
   buffer[index++] = (byte)(alt >> 8);
   buffer[index++] = (byte)alt;
}
Azure IoT Central map position granularity

Before the message can be sent it needs to be converted to its Binary Coded Decimal(BCD) representation and all formatting characters removed.

public string Bcd()
{
   StringBuilder payloadBcd = new StringBuilder(BitConverter.ToString(buffer, 0, index));

   payloadBcd = payloadBcd.Replace("-", "");

   return payloadBcd.ToString();
}

TTN Device Data Display
Visual Studio 2019 Debug output

The implementation had to be revised a couple of times so It would work with desktop and GHI Electronics TinyCLRV2 powered devices. There maybe some modifications required as I port it to nanoFramework and Wilderness Labs Meadow devices.

“Don’t forget to flush” Application Insights Revisited

This post revisits a previous post “Don’t forget to flush” Application insights and shows how to configure the instrumentation key in code or via the ApplicationInsights.config file.

 class Program
   {
      static void Main(string[] args)
      {
#if INSTRUMENTATION_KEY_TELEMETRY_CONFIGURATION
         if (args.Length != 1)
         {
            Console.WriteLine("Usage AzureApplicationInsightsClientConsole <instrumentationKey>");
            return;
         }

         TelemetryConfiguration telemetryConfiguration = new TelemetryConfiguration(args[0]);
         TelemetryClient telemetryClient = new TelemetryClient(telemetryConfiguration);
         telemetryClient.TrackTrace("INSTRUMENTATION_KEY_TELEMETRY_CONFIGURATION", SeverityLevel.Information);
#endif
#if INSTRUMENTATION_KEY_APPLICATION_INSIGHTS_CONFIG
         TelemetryClient telemetryClient = new TelemetryClient();
         telemetryClient.TrackTrace("INSTRUMENTATION_KEY_APPLICATION_INSIGHTS_CONFIG", SeverityLevel.Information);
#endif
         telemetryClient.TrackTrace("This is an AI API Verbose message", SeverityLevel.Verbose);
         telemetryClient.TrackTrace("This is an AI API Information message", SeverityLevel.Information);
         telemetryClient.TrackTrace("This is an AI API Warning message", SeverityLevel.Warning);
         telemetryClient.TrackTrace("This is an AI API Error message", SeverityLevel.Error);
         telemetryClient.TrackTrace("This is an AI API Critical message", SeverityLevel.Critical);

         telemetryClient.Flush();

         Console.WriteLine("Press <enter> to exit");
         Console.ReadLine();
      }

A sample project is available here

Poetry in Klingon

Along time ago I read an article which said “There is no easy way to program in parallel it’s like writing poetry in Klingon”. Little did I know that you can buy bound books of Klingon poetry.

I had noticed odd characters getting displayed every so often, especially when I had many devices working. Initially, I though it was two (or more) of the devices interfering with each other but after looking at the logging the payload CRC was OK

RegIrqFlags 01010000 = RxDone + Validheader (The PayloadCrcError bit is not set)

Received 23 byte message Hello Arduino LoRa! 142
RegIrqFlags 01010000
RX-Done
Received 23 byte message Hello Arduino LoRa! 216
The thread 0xea4 has exited with code 0 (0x0).
The thread 0x1034 has exited with code 0 (0x0).
RegIrqFlags 01010000
RX-Done
Received 23 byte message Ngllo Arduino /R�� �44
RegIrqFlags 01010000
RX-Done
Received 23 byte message Hello Arduino LoRa! 218
RegIrqFlags 01010000
RX-Done

I think the problem is that under load the receive and transmit code are accessing the SX127X FIFO and messing things up or the CRC isn’t getting attached.

I’ll put a lock around where bytes are inserted into and read from the FIFO, check the sequencing of register reads and do some more stress testing.

I turned off sending of messages and still got the corruption.

Then I went back to by Receive Basic example and it still had the problem. Looks like it might be something to do with the way I access the FIFO.

egIrqFlags 01010000
Receive-Message
Received 23 byte message Hello Arduino LoRa! 112
Receive-Done
Receive-Wait
........................
RegIrqFlags 01010000
Receive-Message
Received 23 byte message Hello Arduino LoRa! 110
Receive-Done
Receive-Wait
.....
RegIrqFlags 01110000
Receive-Message
Received 19 byte message Hello NetMFh���u�P
Receive-Done
Receive-Wait
.

 

Azure Meetup Christchurch notes

For the people who came to my Azure meetup session this evening

Sources of sensors and development boards

http://www.adafruit.com
http://www.elecrow.com (watering kits)
http://www.ingenuitymicro.com (NZ based dev boards)
http://www.netduino.com (.NetMF development boards)
http://www.makerfabs.com
http://www.seeedstudio.com
http://www.tindie.com

nRF24Shields for RPI devices
http://www.tindie.com/products/ceech/new-raspberry-pi-to-nrf24l01-shield/

nRF24Shields for *duino devices in AU
embeddedcoolness.com

Raspberry PI Source in CHC
http://www.wavetech.co.nz

RFM69 & LoRa Modules
http://www.wisen.com.au

local sensor and device resellers quick turnaround
http://www.mindkits.co.nz
http://www.nicegear.co.nz

http://www.diyelectricskateboard.com

The watch development platform
http://www.hexiwear.com

http://www.gowifi.co.nz (Antennas & other wireless kit based in Rangiora)

my projects
http://www.hackster.io/KiwiBryn
io.adafruit.com/BrynHLewis/dashboards/home-environment

“Don’t forget to flush” Application Insights

Revisited March 2020

An Azure solution I was working on had a .Net console application which ran on a server at the customer’s premises. It was scheduled task that uploaded some files to azure blob storage every 5 minutes.

To help with debugging I added support for Azure application Insights but after monitoring the application for a while I noticed some shutdown events were not getting uploaded.

Initially I was a bit confused because when I ran the application on my desktop it worked fine (It works on my machine). I found this was because when launched from the debugger the application would upload any files it found then wait until I pressed to exit and this was enough time for the shutdown messages to get uploaded.

The code for a smallest example application is below (I pass the instrumentation key as a command line parameter).

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;

namespace devMobile.Azure.ApplicationInsightsClientConsole
{
   class Program
   {
      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         TelemetryClient telemetryClient = new TelemetryClient();

         telemetryClient.TrackTrace("This is Application Insights native");

         telemetryClient.TrackTrace("Application startup");

         // application does stuff

         telemetryClient.TrackTrace("Application shutdown");

         telemetryClient.Flush();
      }
   }
}

Sample project AzureApplicationInsightsClientConsole