Azure Event Hub Service Gateway V0.1

My Netduino and Arduino devices can’t do https so I had been looking at different approaches for uploading sensor data to a Microsoft Azure Event Hub. In a previous post I published the “simplest” possible useful program (a console application) which could upload data and this code builds on that.

In this proof of concept I have integrated the core of the console application code into an ASP.NET MVC WebAPI 2 project which acts as a service gateway. My Netduino clients now use a website hosted on my Essentials 2012 home server to forward the requests to a Microsoft Azure Event Hub .

For more detail about how to program the energy monitor shield see these posts about the Nokia 5110 display, nrf24L01 wireless link, and non invasive current sensor algorithm optimisations.

try
{
   using (HttpWebRequest request = (HttpWebRequest)WebRequest.Create( AzureGatewayUrl ))
   {
      string payload = @"{""DeviceId"":" + deviceId + @",""Usage"":" + value + "}";
      byte[] buffer = Encoding.UTF8.GetBytes(payload);
      request.Method = "POST";
      request.ContentLength = buffer.Length;
      request.ContentType = "text/csv";
      request.KeepAlive = false;
      request.Timeout = 5000;
      request.ReadWriteTimeout = 5000;

      using (Stream stream = request.GetRequestStream())
      {
         stream.Write(buffer, 0, buffer.Length);
      }
      using (var response = (HttpWebResponse)request.GetResponse())
      {
         Debug.Print("HTTP Status" + response.StatusCode + " : " + response.StatusDescription);
      }
   }
}
catch (Exception ex)
{
   Debug.Print(ex.Message);
}
Netduino power consumption monitor

Netduino power consumption monitor

Azure Service Bus Explorer by  Paolo Salvatori is great for debugging and testing Service Bus applications like this.

ServiceBus Explorer Displaying event hub power consumption data

ServiceBus Explorer displaying power consumption data

Bill of materials (prices as at Feb 2015)

The Azure Event Hub Service GatewayV0.1 is pretty basic with, no security, doesn’t have a lot of logging. wouldn’t scale terribly well (Though most home systems wouldn’t have a lot of sensors) and is hosted in Internet Information Server(IIS),

In future posts I’ll fix these limitations and make the service gateway secure, easier to install, configure and operate. But, this proof of concept proves the approach is viable

// POST api/eventhub
 public void Post(HttpRequestMessage request)
      {
         try
         {
            string connectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"];
            string eventHubName = ConfigurationManager.AppSettings["Microsoft.ServiceBus.EventHub"];

            NamespaceManager namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);

            EventHubClient client = EventHubClient.Create(eventHubName);

            EventData data = new EventData(request.Content.ReadAsByteArrayAsync().Result);

            // Set user properties if needed
            data.Properties.Add("UploadedAtUTC", DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ss"));
            data.Properties.Add("UploadedBy", "devMobileAzureEventHubGateway");

            client.Send(data);
         }
         catch (Exception ex)
         {
            Debug.WriteLine("Send failed " + ex.Message);
         }
      }

Code Camp Christchurch 2014

The Hardware

Flash an LED

OutputPort led = new OutputPort(Pins.ONBOARD_LED, false);
while ( true)
{
   Led.Write(!Led.Read())
   Thread.Sleep(500)
}

Digital Input – Polled

InputPort button = new InputPort(Pins.ONBOARD_SW1, false, Port.ResistorMode.Disabled);
OutputPort led = new OutputPort(Pins.ONBOARD_LED, false);
while (true)
{
   led.Write(button.Read());
   Thread.Sleep(1000);
}

Digital Input – Interrupt

static OutputPort interuptled = new OutputPort(Pins.ONBOARD_LED, false);
InterruptPort button = new InterruptPort(Pins.ONBOARD_SW1, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeHigh);
button.OnInterrupt += new NativeEventHandler(button_OnInterrupt);

Thread.Sleep(Timeout.Infinite);

static void button_OnInterrupt(uint data1, uint data2, DateTime time)
{
   interuptled.Write(!interuptled.Read());
}

Analog Input

AnalogInput Sensor = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);
while ( true)
{
   Debug.Print( "Value " + Sensor.Read().ToString("F2"));
   Thread.Sleep(500);
}

Pulse Width Modulation Output

AnalogInput brightness = new AnalogInput(AnalogChannels.ANALOG_PIN_A0);
PWM led = new PWM(PWMChannels.PWM_PIN_D5, 1000, 0.0, false);

led.Start();
while (true)
{
   Debug.Print("Brightness " + led.DutyCycle.ToString("F2"));
   led.DutyCycle = brightness.Read();
   Thread.Sleep(500);
}
led.Stop();

Power Consumption Monitor

Developing the software for the Energy Monitor Shield

Robot

Developing the software

  • Determine the distance to objects
  • Control the speed & direction of the motors using a Motor Shield Driver
  • Basic obstacle avoidance
  • Avoid obstacles using a state machine
  • Fine tune the motor speeds using a rotary encoder
  • Connect the GPS
  • Upload the position information to Xively

Heart Rate Monitor

Developing the software

  • Read the buttons using an AnalogInput
  • Count the number of button presses using an InterruptPort and a Timer
  • Determine the pulse rate in BPM by counting
  • Determine the average pulse rate in BPM
  • Display and manage the pulse rate info on the DFRobot 16×2 Lcd Shield
  • Upload the pulse rate information to xively

 

Energy Monitor Shield Nokia 5100 Display

The Energy Monitor Shield is supplied with a Nokia 5110 LCD for displaying instantaneous power consumption etc. There is an excellent Netduino driver for the Nokia 5100 display by omar, and with a few modifications this works with the Energy Monitor Shield. I removed the backlight support and made a few other simple modifications.

Nokia_5110 Lcd = new Nokia_5110(Pins.GPIO_PIN_D3, Pins.GPIO_PIN_D6, Pins.GPIO_PIN_D5);

while (true)
{
   Lcd.Clear();
   Lcd.DrawString(0, 0, DateTime.Now.ToString("HH:mm:ss"), true);
   Lcd.DrawString(0, 1, DateTime.Now.ToString("HH:mm:ss"), true);
   Lcd.DrawString(0, 2, DateTime.Now.ToString("HH:mm:ss"), true);
   Lcd.Refresh();
   Thread.Sleep(500)
}

Netduino Nokia 5110 driver

Energy Monitor Shield Noise Reduction

In a couple of previous posts the noise on the Netduino AnalogInput and the impact of this on the RMS current measurement was discussed. I trialled two versions of the code to see if my approach worked. Both versions used the initial calibration phase to measure the maximum and minimum values of the noise.

int valueSum = 0;
int valueNoiseMinimum = int.MaxValue;
int valueNoiseMaximum = int.MinValue;
int valueSumSqr = 0;
int offset;
AnalogInput x1 = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);

// Calculate the sum for offset for first run
for (int i = 0; i < SampleCount; i++)
{
  int value = x1.ReadRaw();
  valueSum = valueSum + value;

   if (value < valueNoiseMinimum)
   {
      valueNoiseMinimum = value;
   }
   if (value > valueNoiseMaximum)
   {
      valueNoiseMaximum = value;
   }
}

offset = valueSum / SampleCount;
valueNoiseMinimum -= offset;
valueNoiseMaximum -= offset;

The first version used only the initial offset

Stopwatch stopwatch = Stopwatch.StartNew();
stopwatch.Start();

for (int i = 0; i < SampleCount; i++)
{
   int value = x1.ReadRaw();
   value -= offset;

   if ((value &gt; valueNoiseMaximum) || (value &lt; valueNoiseMinimum))
   {
   valueSumSqr += (value * value);
   }
}
stopwatch.Stop();

RMS 42.2729 RMS Current 3.4A RMS Power 775W Duration = 3301 mSec 30293/sec
RMS 42.2137 RMS Current 3.4A RMS Power 774W Duration = 3302 mSec 30284/sec
RMS 42.2374 RMS Current 3.4A RMS Power 775W Duration = 3302 mSec 30284/sec
RMS 42.1307 RMS Current 3.4A RMS Power 773W Duration = 3302 mSec 30284/sec
RMS 42.1307 RMS Current 3.4A RMS Power 773W Duration = 3302 mSec 30284/sec
RMS 42.1189 RMS Current 3.4A RMS Power 773W Duration = 3302 mSec 30284/sec
RMS 42.1307 RMS Current 3.4A RMS Power 773W Duration = 3302 mSec 30284/sec
RMS 42.1070 RMS Current 3.4A RMS Power 772W Duration = 3302 mSec 30284/sec
RMS 42.1189 RMS Current 3.4A RMS Power 773W Duration = 3302 mSec 30284/sec
RMS 42.1426 RMS Current 3.4A RMS Power 773W Duration = 3303 mSec 30275/sec

The second version updated the offset every iteration

Stopwatch stopwatch = Stopwatch.StartNew();
stopwatch.Start();

for (int i = 0; i &lt; SampleCount; i++)
{
   int value = x1.ReadRaw();
   valueSum += value;
   value -= offset;

   if ((value &gt; valueNoiseMaximum) || (value &lt; valueNoiseMinimum))
   {
      valueSumSqr += (value * value);
   }
}
stopwatch.Stop();
offset = valueSum / SampleCount;

This was slightly slower due to the extra addition operation in the sampling loop

RMS 41.5933 RMS Current 3.3A RMS Power 763W Duration = 3537 mSec 28272/sec
RMS 41.6653 RMS Current 3.3A RMS Power 764W Duration = 3541 mSec 28240/sec
RMS 41.6053 RMS Current 3.3A RMS Power 763W Duration = 3538 mSec 28264/sec
RMS 41.5572 RMS Current 3.3A RMS Power 762W Duration = 3537 mSec 28272/sec
RMS 41.5572 RMS Current 3.3A RMS Power 762W Duration = 3537 mSec 28272/sec
RMS 41.5331 RMS Current 3.3A RMS Power 762W Duration = 3537 mSec 28272/sec
RMS 41.4970 RMS Current 3.3A RMS Power 761W Duration = 3540 mSec 28248/sec
RMS 41.4849 RMS Current 3.3A RMS Power 761W Duration = 3538 mSec 28264/sec
RMS 41.4849 RMS Current 3.3A RMS Power 761W Duration = 3538 mSec 28264/sec
RMS 41.4849 RMS Current 3.3A RMS Power 761W Duration = 3516 mSec 28441/sec

At 28K4 samples per second the self adjusting RMS calculation is sampling the 50Hz waveform much more frequently than required.

Energy Monitor Shield RMS Calculation

The voltage output by the current sensor and measured by the Netduino needs to be corrected using the offset value then the RMS value calculated. This RMS value then needs to be adjusted taking into account the voltage range of the Netduino analog input (0V-3V3), the resolution of the analog input (12 bits) and the voltage output by the non-invasive current sensor (0~1V for 0~30A).

My approach appears to produce reasonable values but I will need to compare them with a calibrated reference device to check its accuracy. The 18W measurement with no current flowing is due to the noise on the analog input discussed in an earlier post.

The first version of the software used the initial offset value, the second version updates the offset value at the end of each set of samples.

int valueSum = 0;
int valueSumSqr = 0;
int offset;
AnalogInput x1 = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);

// Calculate the sum for initial offset
for (int i = 0; i < SampleCount; i++)
{
valueSum += x1.ReadRaw();
}
offset = valueSum / SampleCount;

Stopwatch stopwatch = Stopwatch.StartNew();
stopwatch.Start();

for (int i = 0; i < SampleCount; i++)
{
int value = x1.ReadRaw();

value -= offset;

valueSumSqr += (value * value);
}
stopwatch.Stop();

double rms = System.Math.Sqrt((double)(valueSumSqr / SampleCount));
double rmsCurrent = rms * (3.3 / 4096.0) * 3.3 * 30.0;
double rmsWatts = rmsCurrent * 230;

Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2588 mSec 38639/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2587 mSec 38654/sec RMS 1.0 RMS Current 0.1A RMS Power 18W

int valueSum = 0;
int valueSumSqr = 0;
int offset;
AnalogInput x1 = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);
// Calculate the sum for initial offset
for (int i = 0; i < SampleCount; i++)
{
valueSum += x1.ReadRaw();
}
offset = valueSum / SampleCount;

Stopwatch stopwatch = Stopwatch.StartNew();
stopwatch.Start();

for (int i = 0; i < SampleCount; i++)
{
int value = x1.ReadRaw();

valueSum += value;

value -= offset;

valueSumSqr += (value * value);
}
stopwatch.Stop();

offset = valueSum / SampleCount;

double rms = System.Math.Sqrt((double)(valueSumSqr / SampleCount));
double rmsCurrent = rms * (3.3 / 4096.0) * 3.3 * 30.0 ;
double rmsWatts = rmsCurrent * 230;

Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W
Duration = 2816 mSec 35511/sec RMS 1.0 RMS Current 0.1A RMS Power 18W

Both versions appear to sample the output of the non-invasive current sensor at a more than sufficient rate.

Energy Monitor Shield Analog Input Noise

While writing the calibration code I noticed that the voltage reading was a bit noisy so I modified the code to record the minimum & maximum values then put the current sensor clamp on a wire not carrying any current.

int value;
int valueSum = 0;
int valueMinimum = int.MaxValue;
int valueMaximum = int.MinValue;
AnalogInput x1 = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);

stopwatch.Start();
for (int i = 0; i < SampleCount; i++)
{
value = x1.ReadRaw();

if (value < valueMinimum)
{
valueMinimum = value;
}
if (value > valueMaximum)
{
valueMaximum = value;
}
valueSum += value;
}
stopwatch.Stop();

Duration = 3509 mSec 28498/sec Min=2031 Max=2052 Avg=2041
Duration = 3501 mSec 28563/sec Min=2031 Max=2052 Avg=2041
Duration = 3500 mSec 28571/sec Min=2031 Max=2053 Avg=2041
Duration = 3501 mSec 28563/sec Min=2031 Max=2053 Avg=2041
Duration = 3500 mSec 28571/sec Min=2031 Max=2051 Avg=2041
Duration = 3500 mSec 28571/sec Min=2031 Max=2052 Avg=2041
Duration = 3500 mSec 28571/sec Min=2031 Max=2053 Avg=2041
Duration = 3501 mSec 28563/sec Min=2032 Max=2053 Avg=2041
Duration = 3500 mSec 28571/sec Min=2031 Max=2053 Avg=2041
Duration = 3500 mSec 28571/sec Min=2030 Max=2052 Avg=2041

Looks like there is a little bit of noise on the input when there is no current flowing.

I also tried using the baked in Min & Max functions but these were a bit slower which was not what I was expecting

int value;
int valueSum = 0;
int valueMinimum = int.MaxValue;
int valueMaximum = int.MinValue;
AnalogInput x1 = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);

stopwatch.Start();
for (int i = 0; i < SampleCount; i++)
{
value = x1.ReadRaw();

valueMinimum = System.Math.Min(value, valueMinimum);
valueMaximum = System.Math.Max(value, valueMaximum);

valueSum += value;
}
stopwatch.Stop();

Duration = 4672 mSec 21390/sec Min=2036 Max=2048 Avg=2041
Duration = 4665 mSec 21436/sec Min=2036 Max=2049 Avg=2041
Duration = 4665 mSec 21436/sec Min=2035 Max=2049 Avg=2041
Duration = 4664 mSec 21440/sec Min=2036 Max=2048 Avg=2041
Duration = 4664 mSec 21440/sec Min=2036 Max=2048 Avg=2041
Duration = 4664 mSec 21440/sec Min=2036 Max=2049 Avg=2041
Duration = 4664 mSec 21440/sec Min=2035 Max=2048 Avg=2041
Duration = 4664 mSec 21440/sec Min=2035 Max=2048 Avg=2041
Duration = 4664 mSec 21440/sec Min=2035 Max=2049 Avg=2041
Duration = 4664 mSec 21440/sec Min=2035 Max=2048 Avg=2041

Energy Monitor Shield Sensor Calibration

The Netduino 2 Plus AnalogInput has a range of 0V to 3V3. The Non Invasive Current Sensor we are using returns 0-1V AC for 0-30 Amps AC. To measure the sensor’s output waveform the Energy Monitor Shield uses a voltage divider to offset the reference voltage to 3v3/2. To calibrate the sensor we averaged the input voltage and over 100,000 readings.

The initial code looked like this

int value;
int valueSum = 0;
AnalogInput x1 = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);

stopwatch.Start();
for (int i = 0; i < SampleCount; i++)
{
value = x1.ReadRaw();

valueSum = valueSum + value;
}
stopwatch.Stop();

Duration = 2272 mSec 44014/sec
Duration = 2273 mSec 43994/sec
Duration = 2272 mSec 44014/sec
Duration = 2272 mSec 44014/sec
Duration = 2272 mSec 44014/sec
Duration = 2272 mSec 44014/sec
Duration = 2272 mSec 44014/sec
Duration = 2272 mSec 44014/sec
Duration = 2272 mSec 44014/sec
Duration = 2272 mSec 44014/sec

I then modified the code to not use a temporary variable

int valueSum = 0;
AnalogInput x1 = new AnalogInput(Cpu.AnalogChannel.ANALOG_0);

stopwatch.Start();
for (int i = 0; i < SampleCount; i++)
{
valueSum = valueSum + x1.ReadRaw();
}
stopwatch.Stop();

Duration = 2181 mSec 45829/sec
Duration = 2182 mSec 45829/sec
Duration = 2181 mSec 45829/sec
Duration = 2181 mSec 45829/sec
Duration = 2181 mSec 45829/sec
Duration = 2181 mSec 45829/sec
Duration = 2181 mSec 45829/sec
Duration = 2181 mSec 45829/sec
Duration = 2181 mSec 45829/sec
Duration = 2181 mSec 45829/sec

The code without the temporary variable was slightly faster.