Azure Function Log4Net configuration

This post was inspired by the couple of hours lost from my life yesterday while I figured out how to get Apache Log4Net and Azure Application Insights working in an Azure Function built with .Net Core 2.X.

After extensive searching I found a couple of relevant blog posts but these had complex approaches and I wanted to keep the churn in the codebase I was working on to an absolute minimum.

With the different versions of the libraries involved (Late March 2019) this was what worked for me so YMMV. To provide the simplest possible example I have created a TimerTrigger which logs information via Log4Net to Azure Application Insights.

Initially the Log4Net configuration wasn’t loaded because its location is usually configured in the AssemblyInfo.cs file and .Net Core 2.x code doesn’t have one.

// You can specify all the values or you can default the Build and Revision Numbers
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]
[assembly: log4net.Config.XmlConfigurator]

I figured I would have to manually load the Log4Net configuration and had to look at the file system of machine running the function to figure out where the Log4Net XML configuration file was getting copied to.

The “Copy to output directory” setting is important

Then I had to get the Dependency Injection (DI) framework to build an ExecutionContext for me so I could get the FunctionAppDirectory to combine with the Log4Net config file name. I used Path.Combine which is more robust and secure than manually concatenating segments of a path together.

/*
    Copyright ® 2019 March devMobile Software, All Rights Reserved
 
    MIT License
...
*/
namespace ApplicationInsightsAzureFunctionLog4NetClient
{
	using System;
	using System.IO;
	using System.Reflection;
	using log4net;
	using log4net.Config;
	using Microsoft.ApplicationInsights.Extensibility;
	using Microsoft.Azure.WebJobs;

	public static class ApplicationInsightsTimer
	{
		[FunctionName("ApplicationInsightsTimerLog4Net")]
		public static void Run([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ExecutionContext executionContext)
		{
			ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

			TelemetryConfiguration.Active.InstrumentationKey = Environment.GetEnvironmentVariable("InstrumentationKey", EnvironmentVariableTarget.Process);

			var logRepository = LogManager.GetRepository(Assembly.GetEntryAssembly());
			XmlConfigurator.Configure(logRepository, new FileInfo(Path.Combine(executionContext.FunctionAppDirectory, "log4net.config")));

			log.Debug("This is a Log4Net Debug message");
			log.Info("This is a Log4Net Info message");
			log.Warn("This is a Log4Net Warning message");
			log.Error("This is a Log4Net Error message");
			log.Fatal("This is a Log4Net Fatal message");

			TelemetryConfiguration.Active.TelemetryChannel.Flush();
		}
	}
}

Log4Net logging in Azure Application Insights

The latest code for my Azure Function Log4net to Applications Insights sample along with some samples for other logging platforms is available on GitHub.

NLog and Application Insights

Another of my clients has an application which uses NLog and sooner or later they are going to want to move their logging to Azure Application Insights.

The application consists of a number of Azure websites and some embedded clients. The Azure applications log information to the local file system on each box but the number of boxes is growing so finding and tracing issues is becoming painful.

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights.Extensibility;
using NLog;

namespace devMobile.Azure.ApplicationInsightsNLogClient
{
   class Program
   {
      private static Logger log = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType.ToString());

      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         log.Trace("This is nLog");
         log.Debug("This is a Debug message");
         log.Info("This is a Info message");
         log.Warn("This is a Warning message");
         log.Error("This is an Error message");
         log.Fatal("This is a Fatal message");

         new Microsoft.ApplicationInsights.TelemetryClient().Flush();
      }
   }
}

Sample code ApplicationInsightsNLogClient

Two clients to go, one which uses serilog the other has a DIY system which I’m ignoring as long as possible.

Apache Log4net and Application Insights

One of my clients had built a Fintech application which uses Apache log4net and I have just finished moving the logging to Azure Application Insights.

The application used to run on a couple of dedicated servers but its core processing engine is now run on an Azure Virtual Machine Scale Set(VMSS).

The application used to log information to the local file system on each server but with more machines (upto a dozen) and them being started up and shutdown in response to customer demand this wasn’t a viable approach.

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using log4net;
using Microsoft.ApplicationInsights.Extensibility;

namespace devMobile.Azure.ApplicationInsightsLog4NetClient
{
   class Program
   {
      public static ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         log.Info("This is Log4net");
         log.Debug("This is a Debug message");
         log.Info("This is a Info message");
         log.Warn("This is a Warning message");
         log.Error("This is an Error message");
         log.Fatal("This is a Fatal message");

         new Microsoft.ApplicationInsights.TelemetryClient().Flush();
      }
   }
}

Sample code ApplicationInsightsLog4NetClient

Microsoft Enterprise Library and Application Insights

One of my clients has a largish application (120+ projects) which uses the Microsoft Patterns and Practices Enterprise Library V6 data access, exception handling, logging and transient fault handling blocks.

To get consistent logging across Classic Cloud services, Azure websites and Azure functions etc. we are in the process of moving all our diagnostics to Azure application insights.

My proof of concept uses a community developed Enterprise Library listener and it appears to be working well.

Beware the Visual Studio configuration tool plug-in rewrites and application config file removing the application insights enterprise library trace listener setup.

The code for a smallest example application is below (I pass the instrumentation key as a command line parameter).

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.Practices.EnterpriseLibrary.Logging;
using Microsoft.Practices.EnterpriseLibrary.ExceptionHandling;
namespace ApplicationInsightsEnterpriseLibraryClient
{
   class Program
   {
      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         LogWriterFactory logWriterFactory = new LogWriterFactory();
         LogWriter logWriter = logWriterFactory.Create();
         Logger.SetLogWriter(logWriter);

         ExceptionManager exceptionManager = new ExceptionPolicyFactory().CreateManager();
         ExceptionPolicy.SetExceptionManager(exceptionManager);

         logWriter.Write("This is Entlib", "General");

         logWriter.Write("Application startup", "Startup");

         logWriter.Write("General category", "General");
         logWriter.Write(new LogEntry() { Severity = System.Diagnostics.TraceEventType.Error, Categories = { "General" }, Message = "General category more complex overload", Title = "Dumpster fire" });

         try
         {
            throw new ApplicationException("Something bad has happened");
         }
         catch (Exception ex)
         {
            bool rethrow = ExceptionPolicy.HandleException(ex, "ProgramMain");
            if (rethrow)
               throw;
         }

         logWriter.Write("Application shutdown", "Shutdown");

         new Microsoft.ApplicationInsights.TelemetryClient().Flush();
      }
   }
}

Sample project ApplicationInsightsEnterpriseLibraryClient

Thanks to bveerendrakumar for sharing your code

“Don’t forget to flush” Application Insights

An Azure solution I was working on had a .Net console application which ran on a server at the customer’s premises. It was scheduled task that uploaded some files to azure blob storage every 5 minutes.

To help with debugging I added support for Azure application Insights but after monitoring the application for a while I noticed some shutdown events were not getting uploaded.

Initially I was a bit confused because when I ran the application on my desktop it worked fine (It works on my machine). I found this was because when launched from the debugger the application would upload any files it found then wait until I pressed to exit and this was enough time for the shutdown messages to get uploaded.

The code for a smallest example application is below (I pass the instrumentation key as a command line parameter).

//---------------------------------------------------------------------------------
// Copyright (c) 2018, devMobile Software
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//---------------------------------------------------------------------------------
using System;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;

namespace devMobile.Azure.ApplicationInsightsClientConsole
{
   class Program
   {
      static void Main(string[] args)
      {
         if (args.Length != 1)
         {
            Console.WriteLine("Command line argument InstrumentationKey missing");
            return;
         }
         TelemetryConfiguration.Active.InstrumentationKey = args[0];

         TelemetryClient telemetryClient = new TelemetryClient();

         telemetryClient.TrackTrace("This is Application Insights native");

         telemetryClient.TrackTrace("Application startup");

         // application does stuff

         telemetryClient.TrackTrace("Application shutdown");

         telemetryClient.Flush();
      }
   }
}

Sample project AzureApplicationInsightsClientConsole

Enterprise Library V6 Logging with Azure SDK 2.8 and Azure Diagnostics 1.3

In a previous post I wrote about configuring the Enterprise Library V6 to work with Azure Diagnostics. There have been significant changes (detailed in this very helpful post) to the way the Azure Diagnostics infrastructure works for Azure SDK Versions 2.4/2.5. If the diagnostics infrastructure is not properly configured there will be no WADLogs tables created and/or trace information logged.

The following steps provision diagnostics for a Azure web role or worker role. This “cheat sheet” assumes you already have the Azure Service Management Cmdlets installed.

Add-AzureAccount This will prompt for Azure credentials

Get-AzureSubscription –Display details about your subscription(s)

SubscriptionId : 15daec19-f6e9-403c-8652-1234567890123
SubscriptionName : MyCompany
Environment : AzureCloud
DefaultAccount : me@mycompany.co.nz
IsDefault : False
IsCurrent : False
TenantId : e07af3b3-10c2-49a5-97cc-123456789012
CurrentStorageAccountName :
SubscriptionId : eba7ed1c-5503-4349-bcc7-123456789012
SubscriptionName : YourCompany
Environment : AzureCloud
DefaultAccount : you@yourcompany.co.nz
IsDefault : True
IsCurrent : True
TenantId : e07af3b3-10c2-49a5-97cc-1234567890
CurrentStorageAccountName :

If you have more than one Azure subscription you will need to select the one you want to use.

Select-AzureSubscription -Current -SubscriptionName “MyCompany” (beware names are case sensitive)

Get-AzureServiceDisplays a list of your Azure services

ServiceName : myDemoApp
Url : https://management.core.windows.net/eba7ed1c-5503-4349-bcc7-123456789012/services/hostedservices/myDemoApp
Label : myDemoApp
Description :
Location : Australia Southeast
AffinityGroup :
Status : Created
ExtendedProperties : {[ResourceGroup, myDemoApp], [ResourceLocation, Australia Southeast]}
DateModified : 7/01/2016 7:02:44 p.m.
DateCreated : 28/12/2015 6:23:44 p.m.
ReverseDnsFqdn :
WebWorkerRoleSizes : {A5, A6, A7, ExtraLarge, ExtraSmall, Large, Medium, Small, Standard_D1, Standard_D1_v2, Standard_D11, Standard_D11_v2, Standard_D12, Standard_D12_v2, Standard_D13,
Standard_D13_v2, Standard_D14, Standard_D14_v2, Standard_D2, Standard_D2_v2, Standard_D3, Standard_D3_v2, Standard_D4, Standard_D4_v2, Standard_D5_v2}
VirtualMachineRoleSizes : {A5, A6, A7, Basic_A0, Basic_A1, Basic_A2, Basic_A3, Basic_A4, ExtraLarge, ExtraSmall, Large, Medium, Small, Standard_D1, Standard_D1_v2, Standard_D11,
Standard_D11_v2, Standard_D12, Standard_D12_v2, Standard_D13, Standard_D13_v2, Standard_D14, Standard_D14_v2, Standard_D2, Standard_D2_v2, Standard_D3,
Standard_D3_v2, Standard_D4, Standard_D4_v2, Standard_D5_v2}
OperationDescription : Get-AzureService
OperationId : 73d37e69-d3d8-6769-94a2-123456789012
OperationStatus : Succeeded

Get-AzureRole -ServiceName “myDemoApp”

RoleName : WebRole
InstanceCount : 1
DeploymentID : cb4e439907774090be8d123456789012
ServiceName : myDemoApp
OperationDescription : Get-AzureRole
OperationId : 85233c60-f39a-6c01-b51a-123456789012
OperationStatus : Succeeded

RoleName : WorkerRole
InstanceCount : 1
DeploymentID : cb4e439907774090be8d123456789012
ServiceName : myDemoApp
OperationDescription : Get-AzureRole
OperationId : 85233c60-f39a-6c01-b51a-123456789012
OperationStatus : Succeeded

I then modified the role diagnostics config file (diagnostics.wadcfgx) by removing

<DiagnosticsConfiguration xmlns=”http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration”&gt;
+
</DiagnosticMonitorConfiguration>

+

<PrivateConfig xmlns=”http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration”&gt;
<StorageAccount endpoint=”” />
</PrivateConfig>
<IsEnabled>true</IsEnabled> – Not certain about this

I then uploaded it with the following powershell script
$storage_name = “entlib”
$key = “Storage key goes here==”
$config_path=”C:\..\diagnostics.xml”
$service_name=”myDemoApp”
$storageContext = New-AzureStorageContext -StorageAccountName $storage_name -StorageAccountKey $key
Set-AzureServiceDiagnosticsExtension -StorageContext $storageContext -DiagnosticsConfigurationPath $config_path -ServiceName $service_name -Slot Production -Role WebRole

Repeat for WorkerRole and WebRole