Computer Vision in C#

On April 27, 2011, in C#, OpenCV, Programming, Software Development, by LuCuS

CvSharp TestFirst off, let me begin by saying I absolutely love OpenCV. The guys that designed this gem really knew what they were doing. I’ve written several articles showing how to create applications using OpenCV and C#. However, I’m still getting a ton of emails from readers that are still having issues when working with OpenCV and with my schedule, it’s hard to keep track of all of them. And that got me thinking. What would it take to create my own, easier, simpler version of OpenCV using nothing but C#? There wouldn’t be any wrappers or any need for anything that Windows and the .NET framework don’t already provide you. So, I set out to find out.

Before I started writing any code, I did some digging around on the interwebernets to see how other people were doing similar stuff. I began with learning how to connect to a standard USB webcam and capture the video stream from it. Pretty much every article I read talked about using the AVI Capture DLL (avicap32.dll) that comes built into Microsoft Windows. (If you’re curious, you can locate this file in your Windows > System32 folder.) Along with the articles that recommended using avicap32.dll, every single one of those articles had downloads for the same exact source code.

Ok. Cool. I decided to download the code and see what it was all about. After about 2 minutes of tinkering with the examples, I was able to get the application working with my cheap-o USB webcam. However, the performance was pitiful. The video ran like a dead dog, displaying a frame rate of like 5 frames per second if that. So, the video was too slow and had too much of a lag to be of any use for real-time processing like most of us wish to do.

Surely there was something I could do to improve the performance? When I started digging deeper into the code, I noticed a HUGE performance issue. The application was waiting for a set interval and grabbing individual frames from the camera. Then, to make things worse, it was copying those frames onto the Windows clipboard and then turning around and getting the frames from the clipboard and returning thumbnail images back to the Windows form control. Ummm??? WTF?!?!? How could anyone think that was a good idea? I thought maybe it was just that way in the source code example I downloaded. So, I downloaded 4 or 5 more examples from other websites. Each of the examples had their own naming conventions and way of doing things. But, every one of those examples did the same exact thing.

Well, this wasn’t acceptable for what I have in mind. So, I did some playing and rewrote the entire thing from scratch. I’m still using the built-in avicap32.dll, but I did away with all of the clipboard crap, worthless finalizers, and junk that just didn’t need to be there. I have managed to slim down my version of this application to a reduced code base that is less than 40% of the size of the examples I messed around with. I even set it up so that you can pass a handler for your picture box control directly to the API. This cuts out all of the middle-man work and renders the final output directly on your screen. My new version is also 10 times faster than the original. In fact, it’s all real-time now and has zero lag in the video stream.

Capturing video from a camera and displaying it in a Windows form is all I’ve had time to do so far. But, I don’t think it’s going to take me too long to add in many of the features that OpenCV has. I have done a lot of work in this field in the past. So, I already know many of the trade-secrets and how they work. I’m going to mess around with this new framework in between my real work and will hopefully have something usable within the next few weeks. Now, I’m not saying it’s going to have all of the cool stuff that OpenCV has, but I will try to include as much as possible. When I get something ready, I’ll post it on here for you guys to checkout. Heck, I’ll even setup an entirely new website for the project so that I’ll have a place for documentation and discussions solely for the framework. At some point, I’ll even write versions for Java, C++, and Python as well.

In the mean time, I want to invite you guys to help me come up with a name for the framework. Also, share with me any ideas that you may have that could make the framework easier to work with. Is there anything specific about OpenCV that just irritates you? Are there any things that my new framework could do differently to make your lives easier? I already have several ideas that I think could be a major improvement over the current technology. But, I’d still like to see what thoughts you have. I’ll be sure to include recognition both on the new website and inside the framework for those of you that contribute.

While you’re at it, be sure to also think about ideas for GUI tools that could utilize this framework. I would like to create an interface that would allow you to pick from a list how you want the framework to perform and it will just do it. If things work out the way I have in mind, you wouldn’t have to write a single line of code to get the same functionality that many of you are already asking for today. But, for the rest of you that have extremely specific needs for this kind of technology, don’t fret. I still plan on helping you with your current OpenCV applications as well.

So, until next time, happy coding!

Grab yourself a copy of my eBook (Android App Development 101) today!

BUY NOW

Related Posts

Tagged with:  

6 Responses to Computer Vision in C#

  1. AmarjeetAlien says:

    Well,a new genius was born on April 26,2011(You’re already a born Genius).Because it’s always “an idea” that adds to this “Genius-Tree” whose fruits have lead mankind to current status.

    So to start with…few quotes(to let mind go into thinker mode!):

    1. “Everything should be made as simple as possible, but not simpler.”

    2. “We can’t solve problems by using the same kind of thinking we used when we created them.”

    3. “Imagination is more important than knowledge.”

    4. “The only real valuable thing is intuition.”

    -Albert Einstein(all)
    Naming ideas:
    OpenCvX(X for “eXtreme”)
    OpenCvU(ultimate..for you are extending it for all other UI platforms)
    OpenCE(Extended, but I think someone is there with opence.Net name)

    Features:
    1. Try to keep the function/dtatype/methods/etc names and overloading techniques same as original OpenCV. This will give easy portability to programmers.(I’ve implemented so many modules using OpenCV library but porting them to OpenCvSharp is painful as names are different).

    2. thinking….

    Although I’m not skilled enough to make these comments but I believe in the “Power Of Ideas(POI)” and if you can bring people with similar ideas together(look at keyboard “P” “O” “I” all are sits together,its my self-coined term!)….only magic happens. And to get an idea one doesn’t have to be 100 yr old.

    “It just took “an idea” to create this Universe.”-AmarjeetAlien!

    • AmarjeetAlien says:

      Thanks!

    • LuCuS says:

      Awesome! Thanks for the speedy input. Unfortunately, my new framework will be completely different from OpenCV in the sense that it isn’t OpenCV. I’m going to stay away from naming conventions that are similar to OpenCV as I don’t want people to mistake one for the other. I want them to stand apart. Since this is a new framework, there won’t be any easy ways to port existing applications from OpenCV to the new framework. With that said, I do want to simplify the design process in the new framework so that it’s easier and faster to do than it was with OpenCV. I won’t be using any of the OpenCV code in the new framework. Instead, I’ll be writing everything completely from scratch. I’ll even be rewriting the OpenGL stuff from scratch so that I can include support for things like VRML (which is what I used for the bee-flower augmented reality application). Here is some sample code for what I have in mind as far as implementation of the new framework goes: (BTW, I will be using the name CvSharp until we come up with something permanent)

      CvSharp.Device device = CvSharp.DeviceManager.GetAllDevices()[0];
      CvSharp.ImgTemplate template = CvSharp.Image.Load(‘pupil.png’);
      while(true)
      {
      CvSharp.Image img = device.GetCurrentFrame();
      img.DrawRectangle(template);
      pictureBox1.Image = img;
      }
      device.dispose();

      For your application, the code above would be used to locate “pupil.png” within the video feed and draw a rectangle around it. Instead of DrawRectangle, you might want to display the pupil within a separate picture box like so:

      CvSharp.Image img = device.GetCurrentFrame();
      img.DrawRectangle(template);
      pictureBox1.Image = img;
      pictureBox2.Image = img.Locate(template);

      Well, that’s what I have in mind for it so far. But, I’m sure that will change as I head down that road.

  2. AmarjeetAlien says:

    You got to have a team for such wonderful stuffs, which I think you already have. It’ll be really nice for many newbies in OpenCv and CvSharp.

    Meanwhile I’ll keep asking seemingly silly problems and always expecting a positive feedback!

  3. LuCuS says:

    When working on an undertaking like this, it sucks that I’m having to re-invent the wheel. But, I still have my reasons for wanting to write my own computer vision framework. And, I’m pleased with the progress I’ve made over the last couple of days. Things are beginning to come together nicely.

    The more I dig into this, the more respect and admiration I gain for the guys at OpenCV. With every line of code, I gain more insight into the complexities that those guys were challenged with as they’ve progressed.

Leave a Reply