ASP MVC Core V2.1 and Cross-Origin Resource Sharing

I’m working on an project for a customer which implements a number of application programming Interfaces(API) for a Single Page Application(SPA) and other clients. We are using entity tags (ETags) for versioning and the front end developers found the couldn’t access them from javascipt running in mainstream browser clients (June 2018).

The problems was understanding how Cross-Origin Resource Sharing (CORS) worked and how it interacted with our security model (API key and OAuth2.0 depending on the client)

In our scenario we first found the pre-flight check wasn’t working because in the HyperText Transfer Protocol (HTTP) OPTIONS method our X-API-KEY check was failing

OPTIONS http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
Access-Control-Request-Headers: x-api-key
Access-Control-Request-Method: GET
Accept-Encoding: gzip, deflate
Content-Length: 0
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Server: Kestrel
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:48:30 GMT

13
API Key is invalid.
0

So I disabled X-API-KEY validation in startup.cs

public async Task Invoke(HttpContext context)
{
   if (context.Request.Method == "OPTIONS")
   {
      await this.next.Invoke(context);
      return;
   }

   var claims = new List();
…

OPTIONS then worked

OPTIONS http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
Access-Control-Request-Headers: x-api-key
Access-Control-Request-Method: GET
Accept-Encoding: gzip, deflate
Content-Length: 0
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 404 Not Found
Server: Kestrel
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:52:20 GMT
Content-Length: 0

I then turned on CORS allowing pretty much anything

public void ConfigureServices(IServiceCollection services)
{
   services.AddCors(options =>
   {
      options.AddPolicy("CorsPolicy",
      builder => builder.AllowAnyOrigin()
         .AllowAnyMethod()
         .AllowAnyHeader()
         .AllowCredentials());
   });
   services.AddMvc();
}

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
   if (env.IsDevelopment())
   {
      app.UseDeveloperExceptionPage();
   }

   TelemetryConfiguration.Active.InstrumentationKey = this.configuration.GetSection("ApplicationInsights").GetSection("InstrumentationKey").Value;

   loggerFactory.AddLog4Net();
   this.log.Info("Startup.Configure called");

   app.ApplyUserKeyValidation();
   app.UseCors("CorsPolicy");
   app.UseMvc();
   }
}

OPTIONS then worked

OPTIONS http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
Access-Control-Request-Headers: x-api-key
Access-Control-Request-Method: GET
Accept-Encoding: gzip, deflate
Content-Length: 0
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 204 No Content
Vary: Origin
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: x-api-key
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:57:33 GMT

GET then worked

GET http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
X-API-KEY: ABCDEFGHIJKLMNOPQRSTUVWXYZ
Accept-Language: en-NZ
Accept-Encoding: gzip, deflate
If-None-Match: 00-00-00-00-00-00-00-76
Host: xyz.azurewebsites.net
Connection: Keep-Alive

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
ETag: 00-00-00-00-00-00-00-76
Vary: Origin,Accept-Encoding
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:57:34 GMT
Content-Length: 2216

[{"...."}}

But HEAD didn’t work

HEAD http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
X-API-KEY: ABCDEFGHIJKLMNOPQRSTUVWXYZ
Accept-Language: en-NZ
Accept-Encoding: gzip, deflate
If-None-Match: 00-00-00-00-00-00-00-76
Host: xyz.azurewebsites.net
Connection: Keep-Alive

HTTP/1.1 400 Bad Request
Content-Length: 0
Vary: Origin
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 05:59:55 GMT

From the Application Insights logging and RestTest client (which I ran locally and remotely) I could see that the client side code couldn’t access the value of our eTag.  It had to be “exposed”

public void ConfigureServices(IServiceCollection services)
{
   services.AddCors(options =>
   {
      options.AddPolicy("CorsPolicy",
            builder => builder.AllowAnyOrigin()
            .AllowAnyMethod()
            .AllowAnyHeader()
            .WithExposedHeaders("etag")
            .AllowCredentials()
         );
      });
      services.AddMvc();
   }
...

GET then worked

GET http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
X-API-KEY: ABCDEFGHIJKLMNOPQRSTUVWXYZ
ETag: 00-00-00-00-00-00-00-76
Accept-Language: en-NZ
Accept-Encoding: gzip, deflate
If-None-Match: 00-00-00-00-00-00-00-76
Host: xyz.azurewebsites.net
Connection: Keep-Alive

HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
ETag: 00-00-00-00-00-00-00-76
Vary: Origin,Accept-Encoding
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 07:53:41 GMT

[{"...."}}

HEAD then worked

OPTIONS http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
Access-Control-Request-Headers: x-api-key,etag
Access-Control-Request-Method: HEAD
Accept-Encoding: gzip, deflate
Content-Length: 0
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 204 No Content
Vary: Origin
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: x-api-key,etag
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 07:57:31 GMT

HEAD http://xyz.azurewebsites.net/api/portfolio HTTP/1.1
...
X-API-KEY: ABCDEFGHIJKLMNOPQRSTUVWXYZ
ETag: 00-00-00-00-00-00-00-76
Accept-Language: en-NZ
Accept-Encoding: gzip, deflate
Host: xyz.azurewebsites.net
Connection: Keep-Alive
Pragma: no-cache

HTTP/1.1 304 Not Modified
Vary: Origin
Server: Kestrel
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: file://
X-Powered-By: ASP.NET
...
Date: Sun, 24 Jun 2018 07:57:31 GMT

I had some oddness with releasing code updates which I think was down to caching of pre-flight request responses.
Next steps tidy up the headers etc. and lock the CORS configuration down to expose the minimum necessary required for the application to work.

.Net Core & WCF TransportWithMessageCredential

In one of my day jobs I look after a system which has been around since 2010 (Early adopter of Microsoft Azure, developement started on .Net 3.5). The product has a number of Windows Communication Foundation(WCF) services hosted in an Azure CloudService.

A client built with .Net Core wanted to be able to call one of the services which was implemented using wsHttpBinding and TransportWithMessageCredential and this proved a bit more painful than expected…

I first tried the Visual Studio 2017 Microsoft WCF Web Service Reference Provider fromt the WCF Core Team.

The “add connected service” extension dialog allowed me to select an endpoint

ConfigureWCFWebSeriveReference

But the code generation process failed

WCFWebServiceReferenceError.png

The error message wasn’t particularly helpful so I used the command line utility svcutil to generate client classes. Which I used to built a .net core client with and the associated .Net Core WCF NuGet packages.

The console application failed when I called the service with a “PlatformNotSupportedException”. After some searching I found that the .Net Core WCF libraries don’t support TransportWithMessageCredential (September 2017).

Some more searching lead to a StackOverflow article where an answer suggested using the SimpleSOAPClient NuGet package. I then created a new client using the generated classes as the basis for the ones used in my SimpleSOAPClient proof of concept(PoC)

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "4.0.0.0")]
[System.ServiceModel.MessageContractAttribute(WrapperName="Redeem", WrapperNamespace="http://qwertyuiop.com/services2011/08", IsWrapped=true)]
public partial class RedeemRequest
{
    [System.ServiceModel.MessageBodyMemberAttribute(Namespace="http://qwertyuiop.com/services2011/08", Order=1)]
    public string voucherCode;

    [System.ServiceModel.MessageBodyMemberAttribute(Namespace="http://qwertyuiop.com/services2011/08", Order=2)]
    public string merchantId;

    [System.ServiceModel.MessageBodyMemberAttribute(Namespace="http://qwertyuiop.com/services2011/08", Order=3)]
    public string merchantReference;

    [System.ServiceModel.MessageBodyMemberAttribute(Namespace="http://qwertyuiop.com/services2011/08", Order=4)]
    public string terminalId;

    public RedeemRequest()
    {
    }

    public RedeemRequest(string voucherCode, string merchantId, string merchantReference, string terminalId)
    {
        this.voucherCode = voucherCode;
        this.merchantId = merchantId;
        this.merchantReference = merchantReference;
        this.terminalId = terminalId;
    }
}

became

[XmlRoot("Redeem", Namespace = "http://qwertyuiop.com/services2011/08")]
public partial class RedeemRequest
{
   [XmlElement("voucherCode")]
   public string voucherCode;
   [XmlElement("transactionAmount")]
   public decimal transactionAmount;
   [XmlElement("merchantId")]
   public string merchantId;
   [XmlElement("merchantReference")]
   public string merchantReference;
   [XmlElement("terminalId")]
   public string terminalId;
}

This client failed with a SOAPAction related exception so I fired up Telerik Fiddler and found that the header was missing. When I manually added the header in the request composer (after dragging one of my failed requests onto the composer tab) it worked.

I had a look at the code in the SimpleSOAPClient repository to see how to add a custom HTTP Header to a request.

RedeemRequest redeemRequest = new RedeemRequest()
{
   merchantId = "......",
   merchantReference = "......",
   terminalId = "......",
   voucherCode = "......",
};

using (var client = SoapClient.Prepare())
{
   client.HttpClient.DefaultRequestHeaders.Add("SOAPAction", "http://qwertyuiop.com/services2011/08/IRedemptionProxyServiceV1/Redeem");
   var responseEnvelope = await client.SendAsync(
      "https://qwertyuiop.com/RedemptionProxy.svc",
      "https://qwertyuiop.com/services2011/08/IRedemptionProxyServiceV1/Redeem",
      SoapEnvelope.Prepare()
      .WithHeaders(KnownHeader.Oasis.Security.UsernameTokenAndPasswordText(".....", "......"))
      .Body(redeemRequest), ct);

      var response = responseEnvelope.Body<RedeemResponse>();

      Console.WriteLine("Redeem Result:{0}  Message:{1}", response.Result, response.messageText);
   }
}

After sorting out a few typos my request worked as expected. Only a couple of hours lost from my life, hopefully this post will help someone else.

Microsoft Sync Framework timezones

Over the last few months I have been working with the Microsoft Sync Framework and the time zone issues have been a problem.

New Zealand has a 12hr standard time or 13 hr daylight savings time offset from Coordinated Universal Time (UTC) and at a glance our customer data could look ok if treated as either local or UTC.

After some experimentation I found that it was due to Windows Communication Foundation(WCF) serialisation issues (The proposed solutions looks like it might have some limitations, especially across daylight savings time transitions).

For the initial synchronisation DateTime values in the database were unchanged, but for any later incremental synchronisations the DateTime values were adjusted to the timezone of the server (Our Azure Cloud Services are UTC timezone, though I don’t understand why Microsoft by default has them set to US locale with MM/DD/YY date formats)

In our scenario having all of the DateTime values in the cloud local looked like a reasonable option and this article provided some useful insights.

In the end I found that setting the DateSetDateTime  for every DateTime column in each DataTable in the synchronisation DataSet to unspecified in the ProcessChangeBatch (our code was based on the samples) method meant that no adjustment was applied to the incremental updates

public override void ProcessChangeBatch(ConflictResolutionPolicy resolutionPolicy, ChangeBatch sourceChanges, object changeDataRetriever, SyncCallbacks syncCallbacks, SyncSessionStatistics sessionStatistics)
{
try
{
DbSyncContext context = changeDataRetriever as DbSyncContext;

if (context != null)
{
foreach (DataTable table in context.DataSet.Tables)
{
foreach (DataColumn column in table.Columns)
{
// Switching from UnspecifiedLocal to Unspecified is allowed even after the DataSet has rows.
if ((column.DataType == typeof(DateTime)) && (column.DateTimeMode == DataSetDateTime.UnspecifiedLocal))
{
column.DateTimeMode = DataSetDateTime.Unspecified;
}
}
}
...

Hope this helps someone else

Mikrobus.Net Quail and EthClick

In my second batch of MikroElektronika Mikrobus clicks I had purchased an EthClick to explore the robustness and reliability of the Mikrobus.Net IP Stack.

My first trial application uses the Internet Chuck Norris database (ICNBD) to look up useful “facts” about the movie star.

public static void Main()
{
   EthClick ethClick = new EthClick(Hardware.SocketTwo);

   ethClick.Start(ethClick.GenerateUniqueMacAddress("devMobileSoftware"), "QuailDevice");

   while (true)
   {
      if (ethClick.ConnectedToInternet)
      {
         Debug.Print("Connected to Internet");
         break;
      }
   Debug.Print("Waiting on Internet connection");
   }

   while (true)
   {
      var r = new HttpRequest(@&amp;quot;http://api.icndb.com/jokes/random&amp;quot;);

      r.Headers.Add("Accept", "*/*");

      var response = r.Send();
      if (response != null)
      {
         if (response.Status == "HTTP/1.1 200 OK")
         {
            Debug.Print(response.Message);
         }

      }
      else
      {
         Debug.Print("No response");
      }
      Thread.Sleep(10000);
   }
}

The ran first time and returned the following text

7c
{ "type": "success, "value": { "id": 496, "joke": "Chuck Norris went out of an infinite loop.", "categories": ["nerdy"]}}
0

85
{ "type": "success", "value": { "id": 518, "joke": "Chuck Norris doesn't cheat death. He wins fair and square.", "categories": []}}
0

It looks like the HTTP response parsing is not quite right as each message starts with the length of the message in bytes in hex and the terminating “0”.

Some fun with a Netduino Plus and the icndb

A chat with some co-workers about displaying the status of the team’s Jenkins build process led to bit of research into calling RESTful services and JSON support on NetMF devices. Previously this had required a bit of hand crafted code but now it looks like the library support has matured a bit. I don’t run Jenkins at home so I decided to build a NetduinoPlus client for the internet Chuck Norris database which has a RESTful API.

This API returns Chuck Norris “facts”…

“Chuck Norris doesn’t read books. He stares them down until he gets the information he wants.”
“There is no theory of evolution, just a list of creatures Chuck Norris allows to live.”
“Some people wear Superman pajamas. Superman wears Chuck Norris pajamas.”
“Chuck Norris can slam a revolving door.”

The icndb API returns JSON

{ "type": "success", "value": { "id": 109, "joke": "It takes Chuck Norris 20 minutes to watch 60 Minutes.", "categories": [] } }

I used the NetMF system.HTTP libraries to initiate the request and Json.NetMF to unpack the response. This snippet illustrates how I processed the request/response


using (HttpWebRequest request = (HttpWebRequest)WebRequest.Create(@"http://api.icndb.com/jokes/random"))
{
//request.Proxy = proxy;
request.Method = "GET";
request.KeepAlive = false;
request.Timeout = 5000;
request.ReadWriteTimeout = 5000;

using (var response = (HttpWebResponse)request.GetResponse())
{
if (response.StatusCode == HttpStatusCode.OK)
{
byte[] buffer = new byte[response.ContentLength];

using (Stream stream = response.GetResponseStream())
{
stream.Read(buffer, 0, (int)response.ContentLength);

string json = new string(Encoding.UTF8.GetChars(buffer));

Hashtable jsonPayload = JsonSerializer.DeserializeString(json) as Hashtable;

Hashtable value = jsonPayload["value"] as Hashtable ;

Debug.Print(value["joke"].ToString());
}
}
}
}

icndb Netduino client

Bill of materials

HTTPS with NetMF calling an Azure Service Bus endpoint

Back in Q1 of 2013 I posted a sample application which called an Azure Service Bus end point just to confirm that the certificate configuration etc. was good.

Since I published that sample the Azure Root certificate has been migrated so I have created a new project which uses the Baltimore Cyber Trust Root certificate.

The sample Azure ServiceBus client uses a wired LAN connection (original one used wifi module) and to run it locally you will have to update the Date information around line 24.

HTTPS with NetMF HTTP Client managing certificates

One of the services I needed to call from my Fez Spider required an HTTPS connection. The HTTP client sample located at

C:\Users\…\Documents\Microsoft .NET Micro Framework 4.2\Samples\HttpClient

shows how to load a certificate and use it when making a request.

There wasn’t a lot of information about getting the required certificate so I decided to document how I did it. On my Windows Server 2k8 box I use either a web browser or the Certificate Manager for exporting certificates. The easiest way is to use your preferred browser to access the service endpoint (To enable the export functionality you need to “Run as administrator”).

1.IECertificate

View the certificate

2.CertificateDetails

Select the root certificate

3.CertificatePath

View the root certificate information

4.CertificateRoot

View the root certificate details

5.CertificateRootDetails

Export the certificate

6.CertificateRootExport

Save CER file in the resources directory of your NetMF Project and then add it to the application resources.

If you know the Root Certification Authority you can export the certificate using Certificate Manager

Certificate Manager

Don’t forget to Update the SSL Seed using MF Deploy and ensure that the device clock is correct.

I use either an Network Time Protocol (NTP) Client or an RTC (Realtime Clock) module to set the device clock.

Depending on the application and device you might need to set the device clock every so often.