For the last month I have been using preview releases of ML.Net with a focus on Open Neural Network Exchange(ONNX) support. A company I work with has a YoloV5 based solution for tracking the cattle in stockyards so I figured I would try getting YoloV5 working with .Net Core and ML.Net on ARM64.
After some searching I found a repository created by Github user Mentalstack for an ONNX based YoloV5 implementation which I cloned and started hacking. I stared by updating the NuGet packages for the scorer and sample application and fixing what broke.
I didn’t update the System.Drawing.Common Nuget as my Raspberry PI V4 has got .Net Core V5 installed.
The sample application only had one dependency Microsoft.ML.OnnxRuntime which I was able to drop as it was referenced by the YoloV5Net.Scorer.
I then modified the sample application to process all the images in an “input” folder and save the processed images with Minimum Bounding Boxes(MBRs) to the output folder.
using System;
using System.Collections.Generic;
using System.Drawing;
using System.IO;
using Yolov5Net.Scorer;
using Yolov5Net.Scorer.Models;
namespace Yolov5Net.App
{
class Program
{
static void Main(string[] args)
{
var scorer = new YoloScorer<YoloCocoP5Model>("Assets/Weights/yolov5s.onnx");
DateTime startedAtUtc = DateTime.UtcNow;
Console.WriteLine($"{startedAtUtc:yyyy:MM:dd HH:mm:ss} Start");
string[] imageFilesPaths = Directory.GetFiles("Assets/inputs");
foreach (string imageFilePath in imageFilesPaths)
{
using (Image image = Image.FromFile(imageFilePath))
using (Graphics graphics = Graphics.FromImage(image))
{
List<YoloPrediction> predictions = scorer.Predict(image);
foreach (var prediction in predictions) // iterate predictions to draw results
{
double score = Math.Round(prediction.Score, 2);
graphics.DrawRectangles(new Pen(prediction.Label.Color, 1), new[] { prediction.Rectangle });
var (x, y) = (prediction.Rectangle.X - 3, prediction.Rectangle.Y - 23);
graphics.DrawString($"{prediction.Label.Name} ({score})", new Font("Consolas", 16, GraphicsUnit.Pixel), new SolidBrush(prediction.Label.Color), new PointF(x, y));
}
image.Save($"Assets/outputs/{Path.GetFileName(imageFilePath)}");
}
}
DateTime finishedAtUtc = DateTime.UtcNow;
TimeSpan duration = finishedAtUtc - startedAtUtc;
Console.WriteLine($"{finishedAtUtc:yyyy:MM:dd HH:mm:ss} Finish Duration:{duration.TotalMilliseconds}mSec");
}
}
}
The sample images are from wikimedia commons site. Go to Wikimediacommon.md to refer to the image urls and their licenses.
The next step of my Proof of Concept(PoC) was to get the YoloV5 Object Detection sample application working on my Intel(R) Core(TM) i7-8700T CPU @ 2.40GHz 2.40 GHz desktop development system. After debugging the software with Visual Studio I “published” the application to a folder.
The application took roughly 0.9 seconds to process each of my 5 sample images. The next task was to get the YoloV5 sample application working on a Raspberry Pi 4 running Bullseye.
To deploy applications I often copy the contents of the “publish” directory to the device with WinSCP. Getting the Object sample application running on my Raspberry Pi4 took a couple of attempts…
I had forgotten then Unix paths are case sensitive inputs vs. Inputs
pi@raspberrypi4a:~/vsdbg/Yolov5Net.App $ dotnet Yolov5Net.App.dll
Unhandled exception. System.TypeInitializationException: The type initializer for 'Microsoft.ML.OnnxRuntime.NativeMethods' threw an exception.
---> System.DllNotFoundException: Unable to load shared library 'onnxruntime' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libonnxruntime: cannot open shared object file: No such file or directory
at Microsoft.ML.OnnxRuntime.NativeMethods.OrtGetApiBase()
at Microsoft.ML.OnnxRuntime.NativeMethods..cctor()
--- End of inner exception stack trace ---
at Microsoft.ML.OnnxRuntime.SessionOptions..ctor()
at Yolov5Net.Scorer.YoloScorer`1..ctor(String weights, SessionOptions opts) in C:\Users\BrynLewis\source\repos\yolov5-net\src\Yolov5Net.Scorer\YoloScorer.cs:line 326
at Yolov5Net.App.Program.Main(String[] args) in C:\Users\BrynLewis\source\repos\yolov5-net\src\Yolov5Net.App\Program.cs:line 14
Aborted
pi@raspberrypi4a:~/vsdbg/Yolov5Net.App $
\
The ONNX runtime was missing so I confirmed the processor architecture with uname then copied the platform specific file to the application folder with Winscp.
I then checked Yolo V4 Sample application was generating output images with WinSCP.
On the Raspberry PI4B the application took roughly 8.3 seconds to process each of my 5 sample images.
I was “standing on the shoulders of giants” the Mentalstack code just worked, my changes were minimal and largely so I could collect some basic performance metrics. I need to spend some more time figuring out how the implementation works.
devMobile's blog 