ML.Net ONNX Object Detection Sample refactoring

I use CustomVision.AI to tag images, then train, test and tune models for my projects. I wanted to be able to export a model for use on a embedded device with minimal manual steps so the Object Detection-ASP.NET Core Web & WPF Desktop Sample in the dotnet/machine-learning-samples looked like a reasonable place to get some “inspiration”.

Extracting the ObjectDetection-Onnx code from the zip file

I updated the OnnxObjectDectection library, OnnxObjectDetectionApp, and OnnxObjectDetectionWeb project Nugets to the latest versions then “smoke tested” the desktop and web applications.

Updated OnnxObjectDetection project NuGets
Updated OnnxObjectDetectionApp project NuGets

The desktop application used the OpenCvSharp3-AnyCPU NuGet which had been deprecated so I upgraded to OpenCvSharp4-Windows NuGet which required a couple of small code modification.

private async Task CaptureCamera(CancellationToken token)
{
   if (capture == null)
      capture = new VideoCapture();

   capture.Open(0,apiPreference:VideoCaptureAPIs.DSHOW);

   if (capture.IsOpened())
   {
      while (!token.IsCancellationRequested)
      {
         using MemoryStream memoryStream = capture.RetrieveMat().Flip(FlipMode.Y).ToMemoryStream();

         await Application.Current.Dispatcher.InvokeAsync(() =>
         {
            var imageSource = new BitmapImage();

            imageSource.BeginInit();
            imageSource.CacheOption = BitmapCacheOption.OnLoad;
            imageSource.StreamSource = memoryStream;
            imageSource.EndInit();

            WebCamImage.Source = imageSource;
         });

         var bitmapImage = new Bitmap(memoryStream);

         await ParseWebCamFrame(bitmapImage, token);
      }

   capture.Release();
   }
}

I ran the OnnxObjectDetectionApp and the provided TinyYolo2_model.onnx model using my webcam.

TinyYolo2_model identifying me as a “person”
Updated OnnxObjectDetectionWeb project NuGets

I ran the OnnxObjectDetectionWeb with the provided TinyYolo2_model.onnx model and a photograph of a car I used to own.

TinyYolo2_model correctly identifying my Lotus 7 as a car.

I have a simple CustomVision.AI demo project for counting toy farm animals which I used to test my modifications.

Quick test of the ToyCowCounter model in CustomVision.ai portal

I exported The ToyCowCounter in ONNX format

Toy Cow Counter Exporting in ONNX format

I copied the exported file to the OnnxModels folder, and then in the Visual Studio 2019 solution explorer configured the file properties “Build Action-Content” and “Copy To Output Directory-Copy if newer”.

When I restarted the OnnxObjectDetectionApp the application would detect my toy cows with a reasonable accuracy.

ToyCowCounter model identifying a cow

The accuracy of the ToyCowCounter model wasn’t great as it had been trained with a limited dataset collected with a different camera and a plain backdrop.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.