I left a Wisnode Track Lite RAK7200 outside on the deck for a day and the way the positions “snapped” to a grid caught my attention. Based on the size of my property the grid looked to be roughly 10 x 10 meters
“These functions round x downwards to the nearest integer, returning that value as a double. Thus, floor (1.5) is 1.0 and floor (-1.5) is -2.0.”
In the C code the latitude and longitude values are truncated to four decimal places and the altitude to two decimal places. In my C# code I used Math.Round and I wondered what impact that could have…
public void GpsLocationAdd(byte channel, float latitude, float longitude, float altitude)
{
IsChannelNumberValid(channel);
IsBfferSizeSufficient(Enumerations.DataType.Gps);
if ((latitude < Constants.LatitudeMinimum ) || (latitude > Constants.LatitudeMaximum))
{
throw new ArgumentException($"Latitude must be between {Constants.LatitudeMinimum} and {Constants.LatitudeMaximum}", "latitude");
}
if ((latitude < Constants.LongitudeMinimum) || (latitude > Constants.LongitudeMaximum))
{
throw new ArgumentException($"Longitude must be between {Constants.LongitudeMinimum} and {Constants.LongitudeMaximum}", "latitude");
}
if ((altitude < Constants.AltitudeMinimum) || (altitude > Constants.AltitudeMaximum))
{
throw new ArgumentException($"Altitude must be between {Constants.AltitudeMinimum} and {Constants.AltitudeMaximum}", "altitude");
}
int lat = (int)Math.Round(latitude * 10000.0f);
int lon = (int)Math.Round(longitude * 10000.0f);
int alt = (int)Math.Round(altitude * 100.0f);
buffer[index++] = channel;
buffer[index++] = (byte)Enumerations.DataType.Gps;
buffer[index++] = (byte)(lat >> 16);
buffer[index++] = (byte)(lat >> 8);
buffer[index++] = (byte)lat;
buffer[index++] = (byte)(lon >> 16);
buffer[index++] = (byte)(lon >> 8);
buffer[index++] = (byte)lon;
buffer[index++] = (byte)(alt >> 16);
buffer[index++] = (byte)(alt >> 8);
buffer[index++] = (byte)alt;
}
I live in Christchurch New Zealand and the theoretical maximum distance is 13.6 m. So, in summary the LPP latitude and longitude values are most probably fine for tracking applications.
Back in 1986 in my second first year at the University of Canterbury I did “MATH131 Numerical Methods” which was a year of looking at why mathematics in FORTRAN, C, and Pascal sometimes didn’t return the result you were expecting…
public void TemperatureAdd(byte channel, float celsius)
{
if ((index + TemperatureSize) > buffer.Length)
{
throw new ApplicationException("TemperatureAdd insufficent buffer capacity");
}
short val = (short)(celsius * 10);
buffer[index++] = channel;
buffer[index++] = (byte)DataType.Temperature;
buffer[index++] = (byte)(val >> 8);
buffer[index++] = (byte)val;
}
After looking at the code I think the issues was most probably due to the representation of the constant 10(int32), 10.0(double), and 10.0f(single) . To confirm my theory I modified the client to send the temperature with the calculation done with three different constants.
Visual Studio 2019 Debug output windowThe Things Network(TTN) Message Queue Telemetry Transport(MQTT) client
After some trial and error I settled on this C# code for my decoder
public void TemperatureAdd(byte channel, float celsius)
{
if ((index + TemperatureSize) > buffer.Length)
{
throw new ApplicationException("TemperatureAdd insufficent buffer capacity");
}
short val = (short)(celsius * 10.0f);
buffer[index++] = channel;
buffer[index++] = (byte)DataType.Temperature;
buffer[index++] = (byte)(val >> 8);
buffer[index++] = (byte)val;
}
I don’t think this is specifically an issue with the TinyCLR V2 just with number type used for the constant.
I could successfully download raw data to the device but I found that manually unpacking it on the device was painful.
Raw data
I really want to send LPP formatted messages to my devices so I could use a standard LPP library. I initially populated the payload fields in the downlink message JSON. The TTN documentation appeared to indicate this was possible.
Download JSON payload format
Initially I tried a more complex data type because I was looking at downloading a location to the device.
Complex data type
I could see nicely formatted values in the device data view but they didn’t arrive at the device. I then tried simpler data type to see if the complex data type was an issue.
Simple Data Types
At this point I asked a few questions on the TTN forums and started to dig into the TTN source code.
Learning Go on demand
I had a look at the TTB Go code and learnt a lot as I figured out how the “baked in “encoder/decoder worked. I haven’t done any Go coding so it took a while to get comfortable with the syntax. The code my look a bit odd as a Pascal formatter was the closest I could get to Go.
At this point I started to hit the limits of my Go skills but with some trial and error I figured it out…
Executive Summary
The downlink payload values are sent as 2 byte floats with a sign bit, 100 multiplier. The fields have to be named “value_X” where X is is a byte value.
I could see these arrive on my TinyCLR plus RAK811 device and could manually unpack them
The stream of bytes can be decoded on an Arduino using the electronic cats library (needs a small modification) with code this
byte data[] = {0xff,0x38} ; // bytes which represent -2
float value = lpp.getValue( data, 2, 100, 1);
Serial.print("value:");
Serial.println(value);
It is possible to use the “baked” in Cayenne Encoder/Decoder to send payload fields to a device but I’m not certain is this is quite what myDevices/TTN intended.
My implementation was “inspired” by the myDevices C/C++ sample code. The first step was to allocate a buffer to store the byte encoded values. I pre allocated the buffer to try and reduce the impacts of garbage collection. The code uses a manually incremented index into the buffer for performance reasons, plus the inconsistent support of System.Collections.Generic and Language Integrated Query(LINQ) on my three embedded platforms. The maximum length message that can be sent is limited by coding rate, duty cycle and bandwidth of the LoRa channel.
public Encoder(byte bufferSize)
{
if ((bufferSize < BufferSizeMinimum) || ( bufferSize > BufferSizeMaximum))
{
throw new ArgumentException($"BufferSize must be between {BufferSizeMinimum} and {BufferSizeMaximum}", "bufferSize");
}
buffer = new byte[bufferSize];
}
For a simple data types like a digital input a single byte (True or False ) is used. The channel parameter is included so that multiple values of the same data type can be included in a message.
public void DigitalInputAdd(byte channel, bool value)
{
if ((index + DigitalInputSize) > buffer.Length)
{
throw new ApplicationException("DigitalInputAdd insufficent buffer capacity");
}
buffer[index++] = channel;
buffer[index++] = (byte)DataType.DigitalInput;
// I know this is fugly but it works on all platforms
if (value)
{
buffer[index++] = 1;
}
else
{
buffer[index++] = 0;
}
}
For more complex data types like a Global Positioning System(GPS) location (Latitude, Longitude and Altitude) the values are converted to 32bit signed integers and only 3 of the 4 bytes are used.
In part 1 & part 2 I had been ignoring the payload_fields property of the Payload class. The documentation indicates that payload_fields property is populated when an uplink message is Decoded.
I used JSON2Csharp to generate C# classes which would deserialise the above uplink message.
// Third version of classes for unpacking HTTP payload
public class Gps1V3
{
public int altitude { get; set; }
public double latitude { get; set; }
public double longitude { get; set; }
}
public class PayloadFieldsV3
{
public double analog_in_1 { get; set; }
public int digital_in_1 { get; set; }
public Gps1V3 gps_1 { get; set; }
public int luminosity_1 { get; set; }
public double temperature_1 { get; set; }
}
public class GatewayV3
{
public string gtw_id { get; set; }
public ulong timestamp { get; set; }
public DateTime time { get; set; }
public int channel { get; set; }
public int rssi { get; set; }
public double snr { get; set; }
public int rf_chain { get; set; }
public double latitude { get; set; }
public double longitude { get; set; }
public int altitude { get; set; }
}
public class MetadataV3
{
public string time { get; set; }
public double frequency { get; set; }
public string modulation { get; set; }
public string data_rate { get; set; }
public string coding_rate { get; set; }
public List<GatewayV3> gateways { get; set; }
}
public class PayloadV3
{
public string app_id { get; set; }
public string dev_id { get; set; }
public string hardware_serial { get; set; }
public int port { get; set; }
public int counter { get; set; }
public bool is_retry { get; set; }
public string payload_raw { get; set; }
public PayloadFieldsV3 payload_fields { get; set; }
public MetadataV3 metadata { get; set; }
public string downlink_url { get; set; }
}
I added a new to controller to my application which used the generated classes to deserialise the body of the POST from the TTN Application Integration.
[Route("[controller]")]
[ApiController]
public class ClassSerialisationV3Fields : ControllerBase
{
private static readonly ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
public string Index()
{
return "move along nothing to see";
}
[HttpPost]
public IActionResult Post([FromBody] PayloadV3 payload)
{
// Check that the post data is good
if (!this.ModelState.IsValid)
{
log.WarnFormat("ClassSerialisationV3Fields validation failed {0}", this.ModelState.Messages());
return this.BadRequest(this.ModelState);
}
log.Info($"DevEUI:{payload.hardware_serial} Payload Base64:{payload.payload_raw} analog_in_1:{payload.payload_fields.analog_in_1} digital_in_1:{payload.payload_fields.digital_in_1} gps_1:{payload.payload_fields.gps_1.latitude},{payload.payload_fields.gps_1.longitude},{payload.payload_fields.gps_1.altitude} luminosity_1:{payload.payload_fields.luminosity_1} temperature_1:{payload.payload_fields.temperature_1}");
return this.Ok();
}
}
I then updated the TTN application integration to send messages to my new endpoint. In the body of the Application Insights events I could see the devEUI, port, and the payload fields had been extracted from the message.
This arrangement was pretty nasty and sort of worked but in the “real world” would not have been viable. I would need to generate lots of custom classes for each application taking into account the channel numbers (e,g, analog_in_1,analog_in_2) and datatypes used.
I also explored which datatypes were supported by the TTN decoder, after some experimentation (Aug 2019) it looks like only the LPPV1 ones are.
AnalogInput
AnalogOutput
DigitalInput
DigitalOutput
GPS
Accelerometer
Gyrometer
Luminosity
Presence
BarometricPressure
RelativeHumidity
Temperature
What I need is a more flexible way to stored and decode payload_fields property..
As I’m testing my Message Queue Telemetry Transport(MQTT) LoRa gateway I’m building a proof of concept(PoC) .Net core console application for each IoT platform I would like to support.
This PoC was to confirm that I could connect to the myDevices CayenneMQTT API and format the topics and payloads correctly. The myDevices team have built many platform specific libraries that wrap the MQTT platform APIs to make integration for first timers easier (which is great). Though, as an experienced Bring Your Own Device(BYOD) client developer, I did find myself looking at the C/C++ code to figure out how to implement parts of my .Net test client.
The myDevices screen designer had “widgets” which generated commands for devices so I extended the test client implementation to see this worked.
The MQTT broker, username, password, client ID, channel number and optional subscription channel number are command line options.
class Program
{
private static IMqttClient mqttClient = null;
private static IMqttClientOptions mqttOptions = null;
private static string server;
private static string username;
private static string password;
private static string clientId;
private static string channelData;
private static string channelSubscribe;
static void Main(string[] args)
{
MqttFactory factory = new MqttFactory();
mqttClient = factory.CreateMqttClient();
if ((args.Length != 5) && (args.Length != 6))
{
Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID] [Channel]");
Console.WriteLine("[MQTT Server] [UserName] [Password] [ClientID] [ChannelData] [ChannelSubscribe]");
Console.WriteLine("Press <enter> to exit");
Console.ReadLine();
return;
}
server = args[0];
username = args[1];
password = args[2];
clientId = args[3];
channelData = args[4];
if (args.Length == 5)
{
Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} ChannelData:{channelData}");
}
if (args.Length == 6)
{
channelSubscribe = args[5];
Console.WriteLine($"MQTT Server:{server} Username:{username} ClientID:{clientId} ChannelData:{channelData} ChannelSubscribe:{channelSubscribe}");
}
mqttOptions = new MqttClientOptionsBuilder()
.WithTcpServer(server)
.WithCredentials(username, password)
.WithClientId(clientId)
.WithTls()
.Build();
mqttClient.ConnectAsync(mqttOptions).Wait();
if (args.Length == 6)
{
string topic = $"v1/{username}/things/{clientId}/cmd/{channelSubscribe}";
Console.WriteLine($"Subscribe Topic:{topic}");
mqttClient.SubscribeAsync(topic).Wait();
// mqttClient.SubscribeAsync(topic, global::MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce).Wait();
// Thought this might help with subscription but it didn't, looks like ACK might be broken in MQTTnet
mqttClient.ApplicationMessageReceived += MqttClient_ApplicationMessageReceived;
}
mqttClient.Disconnected += MqttClient_Disconnected;
string topicTemperatureData = $"v1/{username}/things/{clientId}/data/{channelData}";
Console.WriteLine();
while (true)
{
string value = "22." + DateTime.UtcNow.Millisecond.ToString();
Console.WriteLine($"Publish Topic {topicTemperatureData} Value {value}");
var message = new MqttApplicationMessageBuilder()
.WithTopic(topicTemperatureData)
.WithPayload(value)
.WithQualityOfServiceLevel(global::MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce)
//.WithQualityOfServiceLevel(MQTTnet.Protocol.MqttQualityOfServiceLevel.ExactlyOnce) // Causes publish to hang
.WithRetainFlag()
.Build();
Console.WriteLine("PublishAsync start");
mqttClient.PublishAsync(message).Wait();
Console.WriteLine("PublishAsync finish");
Console.WriteLine();
Thread.Sleep(30100);
}
}
private static void MqttClient_ApplicationMessageReceived(object sender, MqttApplicationMessageReceivedEventArgs e)
{
Console.WriteLine($"ApplicationMessageReceived ClientId:{e.ClientId} Topic:{e.ApplicationMessage.Topic} Qos:{e.ApplicationMessage.QualityOfServiceLevel} Payload:{e.ApplicationMessage.ConvertPayloadToString()}");
Console.WriteLine();
}
private static async void MqttClient_Disconnected(object sender, MqttClientDisconnectedEventArgs e)
{
Debug.WriteLine("Disconnected");
await Task.Delay(TimeSpan.FromSeconds(5));
try
{
await mqttClient.ConnectAsync(mqttOptions);
}
catch (Exception ex)
{
Debug.WriteLine("Reconnect failed {0}", ex.Message);
}
}
}
For this PoC I used the MQTTnet package which is available via NuGet. It appeared to be reasonably well supported and has had recent updates. There did appear to be some issues with myDevices Cayenne default quality of service (QoS) and the default QoS used by MQTTnet connections and also the acknowledgement of the receipt of published messages.
myDevices Cayenne .Net Core 2 clientCayenne UI with graph, button and value widgets
Overall the initial configuration went ok, I found the dragging of widgets onto the overview screen had some issues (maybe the caching of control settings (I found my self refreshing the whole page every so often) and I couldn’t save a custom widget icon at all.
I put a button widget on the overview screen and associated it with a channel publication. The client received a message when the button was pressed
myDevices .Net Core 2 client displaying a received command message
But the button widget was disabled until the overview screen was manually refreshed.
Overall the myDevices Cayenne experience (April 2019) was a bit flaky with basic functionality like the saving of custom widget icons broken, updates of the real-time data viewer didn’t occur or were delayed, and there were other configuration screen update issues.