NLog and Application Insights

Another of my clients has an application which uses NLog and sooner or later they are going to want to move their logging to Azure Application Insights.

The application consists of a number of Azure websites and some embedded clients. The Azure applications log information to the local file system on each box but the number of boxes is growing so finding and tracing issues is becoming painful.

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights.Extensibility;
using NLog;

namespace devMobile.Azure.ApplicationInsightsNLogClient
{
   class Program
   {
      private static Logger log = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType.ToString());

      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         log.Trace("This is nLog");
         log.Debug("This is a Debug message");
         log.Info("This is a Info message");
         log.Warn("This is a Warning message");
         log.Error("This is an Error message");
         log.Fatal("This is a Fatal message");

         new Microsoft.ApplicationInsights.TelemetryClient().Flush();
      }
   }
}

Sample code ApplicationInsightsNLogClient

Two clients to go, one which uses serilog the other has a DIY system which I’m ignoring as long as possible.

Microsoft Enterprise Library and Application Insights

One of my clients has a largish application (120+ projects) which uses the Microsoft Patterns and Practices Enterprise Library V6 data access, exception handling, logging and transient fault handling blocks.

To get consistent logging across Classic Cloud services, Azure websites and Azure functions etc. we are in the process of moving all our diagnostics to Azure application insights.

My proof of concept uses a community developed Enterprise Library listener and it appears to be working well.

Beware the Visual Studio configuration tool plug-in rewrites and application config file removing the application insights enterprise library trace listener setup.

The code for a smallest example application is below (I pass the instrumentation key as a command line parameter).

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.Practices.EnterpriseLibrary.Logging;
using Microsoft.Practices.EnterpriseLibrary.ExceptionHandling;
namespace ApplicationInsightsEnterpriseLibraryClient
{
   class Program
   {
      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         LogWriterFactory logWriterFactory = new LogWriterFactory();
         LogWriter logWriter = logWriterFactory.Create();
         Logger.SetLogWriter(logWriter);

         ExceptionManager exceptionManager = new ExceptionPolicyFactory().CreateManager();
         ExceptionPolicy.SetExceptionManager(exceptionManager);

         logWriter.Write("This is Entlib", "General");

         logWriter.Write("Application startup", "Startup");

         logWriter.Write("General category", "General");
         logWriter.Write(new LogEntry() { Severity = System.Diagnostics.TraceEventType.Error, Categories = { "General" }, Message = "General category more complex overload", Title = "Dumpster fire" });

         try
         {
            throw new ApplicationException("Something bad has happened");
         }
         catch (Exception ex)
         {
            bool rethrow = ExceptionPolicy.HandleException(ex, "ProgramMain");
            if (rethrow)
               throw;
         }

         logWriter.Write("Application shutdown", "Shutdown");

         new Microsoft.ApplicationInsights.TelemetryClient().Flush();
      }
   }
}

Sample project ApplicationInsightsEnterpriseLibraryClient

Thanks to bveerendrakumar for sharing your code

“Don’t forget to flush” Application Insights

An Azure solution I was working on had a .Net console application which ran on a server at the customer’s premises. It was scheduled task that uploaded some files to azure blob storage every 5 minutes.

To help with debugging I added support for Azure application Insights but after monitoring the application for a while I noticed some shutdown events were not getting uploaded.

Initially I was a bit confused because when I ran the application on my desktop it worked fine (It works on my machine). I found this was because when launched from the debugger the application would upload any files it found then wait until I pressed to exit and this was enough time for the shutdown messages to get uploaded.

The code for a smallest example application is below (I pass the instrumentation key as a command line parameter).

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;

namespace devMobile.Azure.ApplicationInsightsClientConsole
{
   class Program
   {
      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         TelemetryClient telemetryClient = new TelemetryClient();

         telemetryClient.TrackTrace("This is Application Insights native");

         telemetryClient.TrackTrace("Application startup");

         // application does stuff

         telemetryClient.TrackTrace("Application shutdown");

         telemetryClient.Flush();
      }
   }
}

Sample project AzureApplicationInsightsClientConsole

Azure IoT Hub nRF24L01 Windows 10 IoT Core Field Gateway

This project is now live on Hackster.IO and github.com with sample *duino, Devduino and Netduino clients. While building the AdaFruit.IO field gateway, Azure IOT Hub field gateways and sample clients I changed the structure of the message payload and spent a bit of time removing non-core functionality and code.

The diagnostics logging code was refactored several times and after reading this reference on docs.Microsoft.com I settled on the published approach.

I considered using the built in Universal Windows Platform (UWP) application data class but this would have made configuration in the field hard for most of the targeted users school students & IT departments.

I have the application running at my house and it has proved pretty robust, last week I though it had crashed because the telemetry data stopped for about 20 minutes. I had a look at the Device portal and it was because Windows 10 IoT core had downloaded some updates, applied them and then rebooted automatically (as configured).

I put a socket on the Raspberry PI nRF24L01 Shield rather than soldering the module to the board so that I could compare the performance of the Low and High power modules. The antenna end of the high power module tends to droop so I put a small piece of plastic foam underneath to prop them up.

I had code to generate an empty JSON configuration but I removed that as it added complexity compared to putting a sample in the github repository.

I considered using a binary format (the nRF24L01 max message length is 32 bytes) but the code required to make it sufficiently flexible rapidly got out of hand and as most of my devices didn’t have a lot of sensors (battery/solar powered *duinos) and it wasn’t a major hassle to send another message so I removed it.

I need to tidy up the project and remove the unused Visual Assets and have a look at the automated update support.

Enterprise Library V6 Logging with Azure SDK 2.8 and Azure Diagnostics 1.3

In a previous post I wrote about configuring the Enterprise Library V6 to work with Azure Diagnostics. There have been significant changes (detailed in this very helpful post) to the way the Azure Diagnostics infrastructure works for Azure SDK Versions 2.4/2.5. If the diagnostics infrastructure is not properly configured there will be no WADLogs tables created and/or trace information logged.

The following steps provision diagnostics for a Azure web role or worker role. This “cheat sheet” assumes you already have the Azure Service Management Cmdlets installed.

Add-AzureAccount This will prompt for Azure credentials

Get-AzureSubscription –Display details about your subscription(s)

SubscriptionId : 15daec19-f6e9-403c-8652-1234567890123
SubscriptionName : MyCompany
Environment : AzureCloud
DefaultAccount : me@mycompany.co.nz
IsDefault : False
IsCurrent : False
TenantId : e07af3b3-10c2-49a5-97cc-123456789012
CurrentStorageAccountName :
SubscriptionId : eba7ed1c-5503-4349-bcc7-123456789012
SubscriptionName : YourCompany
Environment : AzureCloud
DefaultAccount : you@yourcompany.co.nz
IsDefault : True
IsCurrent : True
TenantId : e07af3b3-10c2-49a5-97cc-1234567890
CurrentStorageAccountName :

If you have more than one Azure subscription you will need to select the one you want to use.

Select-AzureSubscription -Current -SubscriptionName “MyCompany” (beware names are case sensitive)

Get-AzureServiceDisplays a list of your Azure services

ServiceName : myDemoApp
Url : https://management.core.windows.net/eba7ed1c-5503-4349-bcc7-123456789012/services/hostedservices/myDemoApp
Label : myDemoApp
Description :
Location : Australia Southeast
AffinityGroup :
Status : Created
ExtendedProperties : {[ResourceGroup, myDemoApp], [ResourceLocation, Australia Southeast]}
DateModified : 7/01/2016 7:02:44 p.m.
DateCreated : 28/12/2015 6:23:44 p.m.
ReverseDnsFqdn :
WebWorkerRoleSizes : {A5, A6, A7, ExtraLarge, ExtraSmall, Large, Medium, Small, Standard_D1, Standard_D1_v2, Standard_D11, Standard_D11_v2, Standard_D12, Standard_D12_v2, Standard_D13,
Standard_D13_v2, Standard_D14, Standard_D14_v2, Standard_D2, Standard_D2_v2, Standard_D3, Standard_D3_v2, Standard_D4, Standard_D4_v2, Standard_D5_v2}
VirtualMachineRoleSizes : {A5, A6, A7, Basic_A0, Basic_A1, Basic_A2, Basic_A3, Basic_A4, ExtraLarge, ExtraSmall, Large, Medium, Small, Standard_D1, Standard_D1_v2, Standard_D11,
Standard_D11_v2, Standard_D12, Standard_D12_v2, Standard_D13, Standard_D13_v2, Standard_D14, Standard_D14_v2, Standard_D2, Standard_D2_v2, Standard_D3,
Standard_D3_v2, Standard_D4, Standard_D4_v2, Standard_D5_v2}
OperationDescription : Get-AzureService
OperationId : 73d37e69-d3d8-6769-94a2-123456789012
OperationStatus : Succeeded

Get-AzureRole -ServiceName “myDemoApp”

RoleName : WebRole
InstanceCount : 1
DeploymentID : cb4e439907774090be8d123456789012
ServiceName : myDemoApp
OperationDescription : Get-AzureRole
OperationId : 85233c60-f39a-6c01-b51a-123456789012
OperationStatus : Succeeded

RoleName : WorkerRole
InstanceCount : 1
DeploymentID : cb4e439907774090be8d123456789012
ServiceName : myDemoApp
OperationDescription : Get-AzureRole
OperationId : 85233c60-f39a-6c01-b51a-123456789012
OperationStatus : Succeeded

I then modified the role diagnostics config file (diagnostics.wadcfgx) by removing

<DiagnosticsConfiguration xmlns=”http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration”&gt;
+
</DiagnosticMonitorConfiguration>

+

<PrivateConfig xmlns=”http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration”&gt;
<StorageAccount endpoint=”” />
</PrivateConfig>
<IsEnabled>true</IsEnabled> – Not certain about this

I then uploaded it with the following powershell script
$storage_name = “entlib”
$key = “Storage key goes here==”
$config_path=”C:\..\diagnostics.xml”
$service_name=”myDemoApp”
$storageContext = New-AzureStorageContext -StorageAccountName $storage_name -StorageAccountKey $key
Set-AzureServiceDiagnosticsExtension -StorageContext $storageContext -DiagnosticsConfigurationPath $config_path -ServiceName $service_name -Slot Production -Role WebRole

Repeat for WorkerRole and WebRole