Xively GPS Location data upload V2

In the previous post I assembled the xively request XML using a StringBuilder rather than using the XML support available in the NetMF. To use the NetMF XML library I needed to add a reference to the DPWS extensions (MFDpwsExtensions) and change the using statement at the top of the module from System.Text to System.Ext.Xml

static void xivelyFeedUpdate(string ApiKey, string feedId, string channel, double latitude, double longitude, double altitude)
{
byte[] buffer;

using (XmlMemoryWriter xmlwriter = XmlMemoryWriter.Create())
{
xmlwriter.WriteProcessingInstruction("xml", "version=\"1.0\" encoding=\"utf-8\"");
xmlwriter.WriteStartElement("eeml");
xmlwriter.WriteStartElement("environment");
xmlwriter.WriteStartElement("location");

xmlwriter.WriteStartAttribute("domain");
xmlwriter.WriteString("physical");
xmlwriter.WriteEndAttribute();

xmlwriter.WriteStartAttribute("exposure");
xmlwriter.WriteString("outdoor");
xmlwriter.WriteEndAttribute();

xmlwriter.WriteStartAttribute("disposition");
xmlwriter.WriteString("mobile");
xmlwriter.WriteEndAttribute();

xmlwriter.WriteStartElement("name");
xmlwriter.WriteString("Location");
xmlwriter.WriteEndElement();

xmlwriter.WriteStartElement("lat");
xmlwriter.WriteString(latitude.ToString("F5"));
xmlwriter.WriteEndElement();

xmlwriter.WriteStartElement("lon");
xmlwriter.WriteString(longitude.ToString("F5"));
xmlwriter.WriteEndElement();

xmlwriter.WriteStartElement("ele");
xmlwriter.WriteString(altitude.ToString("F1"));
xmlwriter.WriteEndElement();

xmlwriter.WriteEndElement();
xmlwriter.WriteEndElement();
xmlwriter.WriteEndElement();

buffer = xmlwriter.ToArray();
}

try
{
using (HttpWebRequest request = (HttpWebRequest)WebRequest.Create(xivelyApiBaseUrl + feedId + ".xml"))
{
request.Method = "PUT";
request.ContentLength = buffer.Length;
request.ContentType = "text/xml";
request.Headers.Add("X-ApiKey", xivelyApiKey);
request.KeepAlive = false;
request.Timeout = 5000;
request.ReadWriteTimeout = 5000;

// request body
using (Stream stream = request.GetRequestStream())
{
stream.Write(buffer, 0, buffer.Length);                }
using (var response = (HttpWebResponse)request.GetResponse())
{
Debug.Print("HTTP Status:" + response.StatusCode + " : " + response.StatusDescription);
}
}
}
catch (Exception ex)
{
Debug.Print(ex.Message);
}
}

I was expecting the XML libraries to be quite chunky, but on my Netduino Plus 2 there wasn’t a huge size difference, the StringBuilder download was 49K8 bytes and the XMLWiter download was 56K1 bytes.

When I ran the StringBuilder and XMLWriter versions they both had roughly 92K6 bytes of free memory.

Realistically there was little to separate the two implementations

Xively GPS Location data upload V1

For one of the code club projects we looked at the National Marine Electronics Association (NMEA) 0183 output of my iteadStudio GPS Shield + Active Antenna. We used the NetMF Toolbox NMEA GPS processing code with a couple of modifications detailed here.

IteadStudio GPS

IteadStudio GPS shield and Antenna

For another project we had used Xively a “Public Cloud for the Internet of Things”. The Xively API has support for storing the position of a “thing” and it didn’t look like it would take much effort to extend the original GPS demo to trial this. The xively Location & waypoints API is RESTful and supports JSON & XML

void xivelyFeedUpdate(string ApiKey, string feedId, string channel, double latitude, double longitude, double altitude)
{
try
{
using (HttpWebRequest request = (HttpWebRequest)WebRequest.Create(xivelyApiBaseUrl + feedId + ".xml"))
{
StringBuilder payload = new StringBuilder();
payload.Append(@"<?xml version=""1.0"" encoding=""UTF-8""?><eeml><environment><location domain=""physical"" exposure=""outdoor"" disposition=""mobile""><name>Location</name><lat>");
payload.Append(latitude.ToString("F5"));
payload.Append("</lat><lon>");
payload.Append(longitude.ToString("F5"));
payload.Append("</lon><ele>");
payload.Append(altitude.ToString("F1"));
payload.Append("</ele></location></environment></eeml>");

byte[] buffer = Encoding.UTF8.GetBytes(payload.ToString());

request.Method = "PUT";
request.ContentLength = buffer.Length;
request.ContentType = "text/xml";
request.Headers.Add("X-ApiKey", xivelyApiKey);
request.KeepAlive = false;
request.Timeout = 5000;
request.ReadWriteTimeout = 5000;

// request body
using (Stream stream = request.GetRequestStream())
{
stream.Write(buffer, 0, buffer.Length);
}
using (var response = (HttpWebResponse)request.GetResponse())
{
Debug.Print("HTTP Status:" + response.StatusCode + " : " + response.StatusDescription);
}
}
}
catch (Exception ex)
{
Debug.Print(ex.Message);
}
}

The position of the “thing” is displayed like this

Xively poisition

The position of my car

The XML was constructed using a stringbuilder (NetMF 4.2) as this appeared easier/smaller than using the baked in XML functionality.

SeeedStudio Grove GPS IteadStudio Shield Comparison

I use the SeeedStudio Grove system for prototyping and teaching. One of the modules is a Global Positioning System (GPS) unit based on the u-blox 5 engine. To get this unit to work you just plug it into the UART socket on the base shield and load the necessary software onto an Arduino (or compatible) board. My day job is working on Microsoft .Net applications so I use a Netduino Plus or a Netduino plus 2.

SeeedStudio GPS unit

SeeedStudio base shield and GPS unit

The Seeedstudio 4 wire connector system has some advantages but for a couple of projects I was looking at I needed to be able to run on a different serial port and access the one pulse per second output. I had a look at several other vendors and the iteadstudio GPS Shield + Active Antenna which is based on the u-blox 6 engine looked like a reasonable alternative.

IteadStudio GPS

IteadStudio GPS shield and Antenna

After some testing I found that the Iteadstudio GPS shield appears to have a shorter time to first fix after a power cycle, I averaged 10 sets of readings for each device and found that in my backyard it took on average 46sec for the Iteadstudio shield and 55sec for the SeeedStudio device.

Both devices output National Marine Electronics Association (NMEA) 0183 sentences and I use the NetMF Toolbox NMEA code to process the data streams. I have modified the NetMF toolbox code with an additional event handler which empties the serial input buffer if there is an error and one of the validation checks needed to be tweaked as it was possible to get an exception due to an empty string.

Around line 45

this._Uart = newSerialPort(SerialPort, BaudRate);
this._Uart.ErrorReceived += newSerialErrorReceivedEventHandler(_Uart_ErrorReceived);
this._Uart.DataReceived += newSerialDataReceivedEventHandler(_Uart_DataReceived);


void _Uart_ErrorReceived(object sender, SerialErrorReceivedEventArgs e)
{
if (_Uart.IsOpen)
{
_Uart.Flush();
}
}

Around line 323

// Have we received a full line of data?
int Pos = this._Buffer.IndexOf("\r\n");
if (Pos >= 1)
{

With these modifications my Netduino Plus 2 can runs for days at a time without buffer overflows or other issues, you just need to be careful to make your event handlers block for as little time as possible.

I have been looking at building an NetMF NMEA driver which runs on a background thread and doesn’t use any string manipulation methods e.g. string.splt so the garbage collector has less to do, but this will be a topic for a future post.

NetMF Gadgeteer SeeedStudio GPS module driver oddness

A few months ago I built a proof of concept NetMF 4.2 application for a client which uploaded data to a Windows Azure Service. Recently we had been talking about extending the application to include support for tagging of the uploaded data with position information.

I purchased a Gadgeeter GPS Module from SeeedStudio and did a trial integration of the module into the client’s application. Initially it was working but I noticed that after a while (I monitored the application using MF Deploy) it would start displaying buffer overflow messages. I did the usual thing and went looking for “slow” code in the GPS events, removed debug print statements and made sure my application and the GPS were “playing nice” but the problem persisted.

My modifications seemed to make little difference so I downloaded the Microsoft Gadgeeter source code from codeplex so I could have a closer look at how the GPS driver worked. (I also personally would also like the driver to return the HDoP, VDoP or PDoP so there is an indication of the accuracy of the position data)

I noticed that there was a GPSTestApp and fired it up to see what would happen. I connected to my FEZ Spider device using MF Deploy and below is the output. The GPS Test application threw an exception shortly after I started it, then started displaying “Buffer OVFLW”. (Odd thing is I it must still be processing some NMEA1803 strings)

I need to do some digging to figure out what’s going on as initially thought it was my code but as the demo application fails I’m not so sure 🙂

Found debugger!
Create TS.
Loading start at a0e00000, end a0e1383c
Assembly: mscorlib (4.2.0.0)     Assembly: Microsoft.SPOT.Native (4.2.0.0)     Assembly: Microsoft.SPOT.Security.PKCS11 (4.2.0.0)     Assembly: System.Security (4.2.0.0)  Loading Deployment Assemblies.
Attaching deployed file.
Assembly: Microsoft.SPOT.IO (4.2.0.0)  Attaching deployed file.
Assembly: GTM.Seeed.GPS (1.6.0.0)  Attaching deployed file.
Assembly: Microsoft.SPOT.Graphics (4.2.0.0)  Attaching deployed file.
Assembly: Microsoft.SPOT.Hardware (4.2.0.0)  Attaching deployed file.
Assembly: Microsoft.SPOT.Hardware.PWM (4.2.0.1)  Attaching deployed file.
Assembly: GHI.Premium.Hardware (4.2.9.0)  Attaching deployed file.
Assembly: System (4.2.0.0)  Attaching deployed file.
Assembly: Gadgeteer.Serial (2.42.0.0)  Attaching deployed file.
Assembly: Microsoft.SPOT.TinyCore (4.2.0.0)  Attaching deployed file.
Assembly: GHI.Premium.System (4.2.9.0)  Attaching deployed file.
Assembly: GPSTestApp (1.0.0.0)  Attaching deployed file.
Assembly: Microsoft.SPOT.Net (4.2.0.0)  Attaching deployed file.
Assembly: System.IO (4.2.0.0)  Attaching deployed file.
Assembly: GHI.Premium.IO (4.2.9.0)  Attaching deployed file.
Assembly: Gadgeteer (2.42.0.0)  Attaching deployed file.
Assembly: Microsoft.SPOT.Hardware.SerialPort (4.2.0.0)  Attaching deployed file.
Assembly: GHIElectronics.Gadgeteer.FEZSpider (1.1.1.0)  Resolving.
GC: 1msec 28272 bytes used, 7311396 bytes available
Type 0F (STRING              ):     24 bytes
Type 15 (FREEBLOCK           ): 7311396 bytes
Type 17 (ASSEMBLY            ):  28176 bytes
Type 34 (APPDOMAIN_HEAD      ):     72 bytes
GC: performing heap compaction...
Ready.

Using mainboard GHI Electronics FEZSpider version 1.0
Program Started
#### Exception System.InvalidOperationException – 0x00000000 (4) ####
#### Message:
#### System.IO.Ports.SerialPort::set_ReadTimeout [IP: 0009] ####
#### Gadgeteer.Interfaces.Serial::ReadLineProcess [IP: 0015] ####
Uncaught exception
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
Buffer OVFLW
Buffer OVFLW
Buffer OVFLW
Buffer OVFLW
Buffer OVFLW

95 x Buffer OVFLW

Buffer OVFLW
Buffer OVFLW
Buffer OVFLW
Buffer OVFLW
GPS last position NO POSITION FIX age 10675199.02:48:05.4775807
Buffer OVFLW
Buffer OVFLW