This project is read-only.

C# video analytics (object detection): How to implement corner detection

In the Computer Vision technology there are several exciting topics to explore. In this article I am going to introduce you a corner detecting application in C# using my USB camera| for the detection. You will get some detailed information on computer vision and corner detection within it. Then I will describe the C# implementation of this corner detecting application, you will see both the C# code and the GUI code.

1. The technology behind the application
  • Brief introduction of Computer Vision technology
  • What is corner detection used for?

2. C# implementation
  • Explanation of the C# code
  • Explanation of the GUI code

Before we head into the article let me share the tools I used for the implementation with you. To be honest you don’t need too much for this program. You just need to have a USB camera device. Plus Microsoft Visual Studio and .Net Framework, but I’m sure you already knew that. Of course you have to use a camera SDK as well for the implementation.

Now let’s get down to business!

1. The technology behind the application

Brief introduction of Computer Vision technology

In order to give you a proper definition I call Wikipedia for help. Computer vision is a field that includes methods for acquiring, processing, analyzing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions. A theme in the development of this field has been to duplicate the abilities of human vision by electronically perceiving and understanding an image.”

Let me summarize that for you. Basically computer vision technology aims to give the ability to see and sense the world to computers. Technically it makes them see the world just as we do. Interesting, huh?

But we don’t have any time to waste, keep on going and find out what is corner detection and what is it used for.

What is corner detection used for?

Once again check Wikipedia to get a proper definition. Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mosaicing, panorama stitching, 3D modeling and object recognition.”

So corner detection is an important part of image analysis and it can be helpful in several fields, like motion detection, 3D modeling or in object recognition.

Here is the end of the introduction part. Now you know what corner detection is good for. So let’s peek into those codes, shall we?!

2. C# implementation

Explanation of the C# code

When we get the idea to create some kind of application and we got all the necessary tools we start the implementation. First of all we will create the C# code.

Well then let’s see those codes:

For detecting the edges we use the ICornerDetector object of the SDK. After we have created an instance with the help of a static ImageProcessedFactory class, we will be able to detect both on still and moving images.

For still images the Process() method of the instance is necessary to create the outgoing picture. However in case of moving images we have to use the ImageProcesserHandler mediahandler.

The ImageProcesserHandler is a VideoHandler class mediahandler which means that it is VideoReceiver and VideoSender at the same time, which we can reach with an instance of the MedicaConnector class.

This program uses a FrameCapture mediahandler as well.

The next step is the global variables:

There is the Webcamera instance which is responsible for displaying the image of the camera.

Then the MediaConnector instance, which helps us to connect the mediahandlers.

We can see the ImageProcessHandler here as well. It is a mediahandler which runs the instances that implement the IImageProcesser interface which does the image processing on the incoming image of the camera.

The ICornerDetector is an interface which is detecting the edges and processing the image. This is the one instance that implements the IImageProcesser interface.

Then we have the FramceCapture mediahandler here. With this we can set the frequency of the image processing.

VideoViewerWF instance is a GUI tool which is responsible for displaying the video for Windows Forms applications.

The DrawingImageProvider instance is a mediahandler which prepares the image for the VideoViewerWF instances sent by the mediahandlers of the VideoSender class.

Here you can see the global variables as a whole:

   public partial class Form1 : Form
           WebCamera _webCamera;
            MediaConnector _connector;
            ImageProcesserHandler _imageProcesserHandler;
            ICornerDetector _cornerDetector;
            FrameCapture _frameCapture;
            VideoViewerWF _originalView;
            VideoViewerWF _processedView;
            DrawingImageProvider _originalImageProvider;
            DrawingImageProvider _processedImageProvider;
            public Form1()

Now let’s go on with the methods:

The initialization of most of the global variables are done by the Init() method. The instance of the FrameCapture mediahandler is also set here. Plus the ICornerDetector instance is created here with the help of the ImageProcesserFactory which is added to the ImageProcesserHandler instance. We subscribe to the DetectionOccured event which indicates the processing of each and every image by the ICornerDetector.
    void Init()
                _frameCapture = new FrameCapture();
                _webCamera = WebCamera.GetDefaultDevice();
                _connector = new MediaConnector();
                _originalImageProvider = new DrawingImageProvider();
                _processedImageProvider = new DrawingImageProvider();
                _cornerDetector = ImageProcesserFactory.CreateCornerDetector();
                _cornerDetector.DetectionOccurred += _cornerDetector_DetectionOccurred;
                _imageProcesserHandler = new ImageProcesserHandler();

The next method is the SetVideoViewers(). This is the method which prepares and initializes the objects responsible for the video displaying. It defines the VideoViewerWF instances and set their properties. It assigns the proper DrawingImageProvider to the VideoViewerWF instances and adds them to the GUI.
    void SetVideoViewers()
                _originalView = new VideoViewerWF
                    BackColor = Color.Black,
                    Location = new Point(10, 20),
                    Size = new Size(320, 240)
                _processedView = new VideoViewerWF
                    BackColor = Color.Black,
                    Location = new Point(350, 20),
                    Size = new Size(320, 240)

We should mention the InvokeGUIThread() method which operates the GUI.
    void InvokeGUIThread(Action action)

Then here is the InitDetectorFields() method which fills the TextBoxes on the GUI with the InvokeGUIThread() helper method with the actual settings of the ICornerDerector instance.
    void InitDetectorFields()
                InvokeGUIThread(() =>
                    chk_ShowImage.Checked = _cornerDetector.ShowImage;
                    tb_Red.Text = _cornerDetector.DrawColor.R.ToString();
                    tb_Green.Text = _cornerDetector.DrawColor.G.ToString();
                    tb_Blue.Text = _cornerDetector.DrawColor.B.ToString();
                    tb_DrawThickness.Text = _cornerDetector.DrawThickness.ToString();
                    tb_BlockSize.Text = _cornerDetector.BlockSize.ToString();
                    tb_DrawSize.Text = _cornerDetector.DrawSize.ToString();
                    tb_Epsilon.Text = _cornerDetector.Epsilon.ToString();
                    tb_MaxDetectedObjects.Text = 
                    tb_MaxIteration.Text = _cornerDetector.MaxIteration.ToString();
                    tb_MinDistance.Text = _cornerDetector.MinDistance.ToString();
                    tb_QualityLevel.Text = _cornerDetector.QualityLevel.ToString();
                    tb_SearchWindowSizeX.Text = _cornerDetector.SearchWindowSize.Width.ToString();
                    tb_SearchWindowSizeY.Text = _cornerDetector.SearchWindowSize.Height.ToString();
                    tb_ZeroZoneSizeX.Text = _cornerDetector.ZeroZoneSize.Width.ToString();
                    tb_ZeroZoneSizeY.Text = _cornerDetector.ZeroZoneSize.Height.ToString();

The next method is the ConnectWebcam(). This is the method which is responsible for the connection of the proper mediahandler instances with the help of the MediaConnector instance. One of the ImageProvider objects receives the original image of the web camera while the other receives the processed one.

    void ConnectWebcam()
                _connector.Connect(_webCamera, _originalImageProvider);
                _connector.Connect(_webCamera, _frameCapture);
                _connector.Connect(_frameCapture, _imageProcesserHandler);
                _connector.Connect(_imageProcesserHandler, _processedImageProvider);

The Start() method makes the madiahandlers start operating after the initializations are done.
    void Start()

In the next part of the code we can see some settings like:

The BlockSize is the size of the averaging block with which we calculate Eigen value and Eigen vector then corners are detected. Epsilon means that the detection can be finished when on the given image a certain accuracy is reached. MaxDetectedObjects means that at most how many matches should be found. MaxInteration stands for that after how many interations can the detection end on the given frame. Here we have the MinDistance which is the minimum distance between two detectable corners. QualityLevel is a multiplier for the Eigen maxmin with which the acceptable minimal quality level can be determined. The SearchWindowSize is the half of the size of the search window. ZeroZoneSize is the half of the size of the dead region in the middle of the search zone over which the summation in the formula is not done.

By clicking on the Set button belonging to the GroupBox the following even will be called which is responsible for setting the CornerDetector:

    void btn_Set_Click(object sender, EventArgs e)
                InvokeGUIThread(() =>
                    _cornerDetector.BlockSize = Int32.Parse(tb_BlockSize.Text);
                    _cornerDetector.Epsilon = Double.Parse(tb_Epsilon.Text);
                    _cornerDetector.MaxDetectedObjects = Int32.Parse(tb_MaxDetectedObjects.Text);
                    _cornerDetector.MaxIteration = Int32.Parse(tb_MaxIteration.Text);
                    _cornerDetector.MinDistance = Int32.Parse(tb_MinDistance.Text);
                    _cornerDetector.QualityLevel = Double.Parse(tb_QualityLevel.Text);
                    _cornerDetector.SearchWindowSize = new Size(Int32.Parse(tb_SearchWindowSizeX.Text),  Int32.Parse(tb_SearchWindowSizeY.Text));
                    _cornerDetector.ZeroZoneSize = new Size(Int32.Parse(tb_ZeroZoneSizeX.Text),   Int32.Parse(tb_ZeroZoneSizeY.Text));

We have to mention that there are some PostProcess settings. After the detection we can set some features and changes of the outgoing image.

Let’s see these settings:

With the ShowImage we can set whether the original picture is show or only the detected shapes on black background. With the DrawColor we can set the color which highlights the detected objects. Meanwhile the DrawThickness sets the thickness of the highlight and the DrawSize which sets the size of the marking squares. So when we click on the Set button these settings are made and the following event is called:

    void btn_HighlightSet_Click(object sender, EventArgs e)
                InvokeGUIThread(() =>
                    _cornerDetector.ShowImage = chk_ShowImage.Checked;
                    _cornerDetector.DrawColor = Color.FromArgb(Int32.Parse(tb_Red.Text),   Int32.Parse(tb_Green.Text), Int32.Parse(tb_Blue.Text));
                    _cornerDetector.DrawSize = Int32.Parse(tb_DrawSize.Text);
                    _cornerDetector.DrawThickness = Int32.Parse(tb_DrawThickness.Text);

Now let’s get in deeper in the detection part!

After processing every image a DetectionOccured event is called.

    void _cornerDetector_DetectionOccurred(object sender, CornerDetectedEventArgs e)
                InvokeGUIThread(() =>
                    foreach (var info in e.Info)

Within the arguments of this event we can find the list of the possibly detected corners. Here you can query the coordinates of these edges.

    void btn_Set_Click(object sender, EventArgs e)
                InvokeGUIThread(() =>
                    _cornerDetector.BlockSize = Int32.Parse(tb_BlockSize.Text);
                    _cornerDetector.Epsilon = Double.Parse(tb_Epsilon.Text);
                    _cornerDetector.MaxDetectedObjects = Int32.Parse(tb_MaxDetectedObjects.Text);
                    _cornerDetector.MaxIteration = Int32.Parse(tb_MaxIteration.Text);
                    _cornerDetector.MinDistance = Int32.Parse(tb_MinDistance.Text);
                    _cornerDetector.QualityLevel = Double.Parse(tb_QualityLevel.Text);
                    _cornerDetector.SearchWindowSize = new Size(Int32.Parse(tb_SearchWindowSizeX.Text), Int32.Parse(tb_SearchWindowSizeY.Text));
                    _cornerDetector.ZeroZoneSize = new Size(Int32.Parse(tb_ZeroZoneSizeX.Text), Int32.Parse(tb_ZeroZoneSizeY.Text));

Explanation of the GUI code

So far so good. But we are not done yet. We still have to create the GUI.

Before I reveal the codes I brought the definition of the Graphical User Interface to you: “In computing, a graphical user interface (GUI) is a type of interface that allows users to interact with electronic devices through graphical icons and visual indicators such as secondary notation, as opposed to text-based interfaces, typed command labels or text navigation…”

Now with the knowledge of what is that we will create, we can start creating the GUI.

So as I mentioned earlier we will have two parts on the GUI where we can see the image of the camera. One of them will show the original image the other one will display the processed image. Therefore we will have two blocks of codes for them.

Here is the code of the Original image:

    this.label1.AutoSize = true;
    this.label1.Font = new System.Drawing.Font("Microsoft Sans Serif", 8.25F,     System.Drawing.FontStyle.Bold, System.Drawing.GraphicsUnit.Point, ((byte)(238)));
    this.label1.Location = new System.Drawing.Point(30, 265);
    this.label1.Name = "label1";
    this.label1.Size = new System.Drawing.Size(87, 13);
    this.label1.TabIndex = 0;
    this.label1.Text = "Original image";

This one is the code of the Processed image:

    this.label2.AutoSize = true;
    this.label2.Font = new System.Drawing.Font("Microsoft Sans Serif", 8.25F,  System.Drawing.FontStyle.Bold, System.Drawing.GraphicsUnit.Point, ((byte)(238)));
    this.label2.Location = new System.Drawing.Point(370, 265);
    this.label2.Name = "label2";
    this.label2.Size = new System.Drawing.Size(103, 13);
    this.label2.TabIndex = 1;
    this.label2.Text = "Processed image";


On the GIU we will have two GroupBoxes. One is for the Settings the other is for the Highlights.

First let me show you the GroupBox of the Settings part:

   this.groupBox1.Location = new System.Drawing.Point(676, 225);
   this.groupBox1.Name = "groupBox1";
   this.groupBox1.Size = new System.Drawing.Size(258, 348);
   this.groupBox1.TabIndex = 12;
   this.groupBox1.TabStop = false;
   this.groupBox1.Text = "Settings";

Since the GUI is for the interaction with the program we should get a button with which we can do this.

So we create a set button in the following way:

    this.btn_Set.Location = new System.Drawing.Point(189, 319);
    this.btn_Set.Name = "btn_Set";
    this.btn_Set.Size = new System.Drawing.Size(58, 23);
    this.btn_Set.TabIndex = 2;
    this.btn_Set.Text = "Set";
    this.btn_Set.UseVisualStyleBackColor = true;
    this.btn_Set.Click += new System.EventHandler(this.btn_Set_Click);

Now we have this button, the only question is what kind of settings will we have on our GUI?

Well above I mentioned that there are some settings.

In this part we can see the codes of these settings:

There is the code for BlockSize:

    this.tb_BlockSize.Location = new System.Drawing.Point(153, 26);
    this.tb_BlockSize.Name = "tb_BlockSize";
    this.tb_BlockSize.Size = new System.Drawing.Size(87, 20);
    this.tb_BlockSize.TabIndex = 3;

The next one is the Epsilon. Do you remember what is it responsible for? It makes the detection stopped when on the given image a certain accuracy is reached.

    this.tb_Epsilon.Location = new System.Drawing.Point(153, 62);
    this.tb_Epsilon.Name = "tb_Epsilon";
    this.tb_Epsilon.Size = new System.Drawing.Size(87, 20);
    this.tb_Epsilon.TabIndex = 5;

At the MaxDetectedObjects part you can provide a number which will be the amount of the maximum detected edges.

    this.tb_MaxDetectedObjects.Location = new System.Drawing.Point(153, 98);
    this.tb_MaxDetectedObjects.Name = "tb_MaxDetectedObjects";
    this.tb_MaxDetectedObjects.Size = new System.Drawing.Size(87, 20);
    this.tb_MaxDetectedObjects.TabIndex = 6;

Let’s go on with the box of the MaxIteration:

    this.tb_MaxIteration.Location = new System.Drawing.Point(153, 134);
    this.tb_MaxIteration.Name = "tb_MaxIteration";
    this.tb_MaxIteration.Size = new System.Drawing.Size(87, 20);
    this.tb_MaxIteration.TabIndex = 7;

…and the MinDistance:

    this.tb_MinDistance.Location = new System.Drawing.Point(153, 170);
    this.tb_MinDistance.Name = "tb_MinDistance";
    this.tb_MinDistance.Size = new System.Drawing.Size(87, 20);
    this.tb_MinDistance.TabIndex = 8;

Now we get to the ZeroZoneSize and to the SearchWindowSize parts. Both of them have 2 boxes where you can provide the values. For these we will have a ZeroZoneSizeY, SearchWindowSizeY, ZeroZoneSizeX, SearchWindowSizeX parts and the labels for the two parts.

    // tb_ZeroZoneSizeY
    this.tb_ZeroZoneSizeY.Location = new System.Drawing.Point(198, 278);
    this.tb_ZeroZoneSizeY.Name = "tb_ZeroZoneSizeY";
    this.tb_ZeroZoneSizeY.Size = new System.Drawing.Size(42, 20);
    this.tb_ZeroZoneSizeY.TabIndex = 23;
    // tb_SearchWindowSizeY
    this.tb_SearchWindowSizeY.Location = new System.Drawing.Point(198, 242);
    this.tb_SearchWindowSizeY.Name = "tb_SearchWindowSizeY";
    this.tb_SearchWindowSizeY.Size = new System.Drawing.Size(42, 20);
    this.tb_SearchWindowSizeY.TabIndex = 22;
    // label14
    this.label14.AutoSize = true;
    this.label14.Location = new System.Drawing.Point(70, 281);
    this.label14.Name = "label14";
    this.label14.Size = new System.Drawing.Size(77, 13);
    this.label14.TabIndex = 21;
    this.label14.Text = "ZeroZoneSize:";
    // tb_ZeroZoneSizeX
    this.tb_ZeroZoneSizeX.Location = new System.Drawing.Point(153, 278);
    this.tb_ZeroZoneSizeX.Name = "tb_ZeroZoneSizeX";
    this.tb_ZeroZoneSizeX.Size = new System.Drawing.Size(42, 20);
    this.tb_ZeroZoneSizeX.TabIndex = 20;
    // label13
    this.label13.AutoSize = true;
    this.label13.Location = new System.Drawing.Point(44, 245);
    this.label13.Name = "label13";
    this.label13.Size = new System.Drawing.Size(103, 13);
    this.label13.TabIndex = 19;
    this.label13.Text = "SearchWindowSize:";
    // tb_SearchWindowSizeX
    this.tb_SearchWindowSizeX.Location = new System.Drawing.Point(153, 242);
    this.tb_SearchWindowSizeX.Name = "tb_SearchWindowSizeX";
    this.tb_SearchWindowSizeX.Size = new System.Drawing.Size(42, 20);
    this.tb_SearchWindowSizeX.TabIndex = 18;

This was the content of the first GroupBox.

Now let’s see the second one which will be the Highlight part:

This is how the code of the Highlight GroupBox looks like:

    this.groupBox2.Location = new System.Drawing.Point(679, 12);
    this.groupBox2.Name = "groupBox2";
    this.groupBox2.Size = new System.Drawing.Size(258, 207);
    this.groupBox2.TabIndex = 13;
    this.groupBox2.TabStop = false;
    this.groupBox2.Text = "Highlight";

First of all we get a Set button here as well:

    this.btn_HighlightSet.Location = new System.Drawing.Point(186, 178);
    this.btn_HighlightSet.Name = "btn_HighlightSet";
    this.btn_HighlightSet.Size = new System.Drawing.Size(58, 23);
    this.btn_HighlightSet.TabIndex = 19;
    this.btn_HighlightSet.Text = "Set";
    this.btn_HighlightSet.UseVisualStyleBackColor = true;
    this.btn_HighlightSet.Click += new System.EventHandler(this.btn_HighlightSet_Click);

We will have an RGB label with three boxes (Blue, Green, Red). There will be a DrawThickness part, a ShowImage checkbox; there is a part for Detection info and a box for DrawSize.

Here is the code block for this part:

This is the DrawThickness part:

    this.tb_DrawThickness.Location = new System.Drawing.Point(153, 95);
    this.tb_DrawThickness.Name = "tb_DrawThickness";
    this.tb_DrawThickness.Size = new System.Drawing.Size(87, 20);
    this.tb_DrawThickness.TabIndex = 17;

We have a label for RGB. Here there will be three boxes for Red, Green and Blue.

    this.label10.AutoSize = true;
    this.label10.Location = new System.Drawing.Point(74, 63);
    this.label10.Name = "label10";
    this.label10.Size = new System.Drawing.Size(33, 13);
    this.label10.TabIndex = 17;
    this.label10.Text = "RGB:";

    this.tb_Blue.Location = new System.Drawing.Point(198, 59);
    this.tb_Blue.Name = "tb_Blue";
    this.tb_Blue.Size = new System.Drawing.Size(42, 20);
    this.tb_Blue.TabIndex = 16;

    this.tb_Green.Location = new System.Drawing.Point(153, 59);
    this.tb_Green.Name = "tb_Green";
    this.tb_Green.Size = new System.Drawing.Size(42, 20);
    this.tb_Green.TabIndex = 15;

    this.tb_Red.Location = new System.Drawing.Point(108, 59);
    this.tb_Red.Name = "tb_Red";
    this.tb_Red.Size = new System.Drawing.Size(42, 20);
    this.tb_Red.TabIndex = 14;

We will have a ShowImage checkbox:

    this.chk_ShowImage.AutoSize = true;
    this.chk_ShowImage.CheckAlign = System.Drawing.ContentAlignment.MiddleRight;
    this.chk_ShowImage.Location = new System.Drawing.Point(37, 25);
    this.chk_ShowImage.Name = "chk_ShowImage";
    this.chk_ShowImage.Size = new System.Drawing.Size(85, 17);
    this.chk_ShowImage.TabIndex = 14;
    this.chk_ShowImage.Text = "ShowImage:";
    this.chk_ShowImage.UseVisualStyleBackColor = true;

Then the Detection part comes with the Detection info section:

    this.lb_Detection.FormattingEnabled = true;
    this.lb_Detection.Location = new System.Drawing.Point(10, 322);
    this.lb_Detection.Name = "lb_Detection";
    this.lb_Detection.Size = new System.Drawing.Size(660, 251);
    this.lb_Detection.TabIndex = 14;

    this.label12.AutoSize = true;
    this.label12.Font = new System.Drawing.Font("Microsoft Sans Serif", 8.25F,     System.Drawing.FontStyle.Bold, System.Drawing.GraphicsUnit.Point, ((byte)(238)));
    this.label12.Location = new System.Drawing.Point(12, 305);
    this.label12.Name = "label12";
    this.label12.Size = new System.Drawing.Size(91, 13);
    this.label12.TabIndex = 15;
    this.label12.Text = "Detection info:";

Then finally we can see the DrawSize:

    this.label4.AutoSize = true;
    this.label4.Location = new System.Drawing.Point(92, 135);
    this.label4.Name = "label4";
    this.label4.Size = new System.Drawing.Size(55, 13);
    this.label4.TabIndex = 21;
    this.label4.Text = "DrawSize:";

    this.tb_DrawSize.Location = new System.Drawing.Point(153, 131);
    this.tb_DrawSize.Name = "tb_DrawSize";
    this.tb_DrawSize.Size = new System.Drawing.Size(87, 20);
    this.tb_DrawSize.TabIndex = 20;

This is the end of the GUI code. Now check out what we have created.


Figure1: GUI of the application


Well it wasn’t that difficult, was it? You don’t only get to see the codes which build up this app but you gained some background information on Computer Vision and Corner detection as well. You saw the C# codes and the codes of the GUI too. Now you can try to create this corner detecting application with your USB camera in C#.
I hope my description on creating this corner detecting application was detailed enough and encouraged you to give it a shot.
Below at the references part you will find all the pages I used for writing this article and a link to the web camera I was using for this application.


Last edited Feb 12, 2015 at 9:39 AM by simonrobert, version 5