It’s been a while since I’ve worked on any computer vision applications. So, tonight I decided to spend a few minutes to play around and have some fun with OpenCV and C#. I dug up an old augmented reality app I created a while back and threw in some 3D models I found on the web. The code isn’t ready to be shared, but I still thought the test results are pretty cool so far and thought I would share those results with all of you. As soon as I get the code to a stable point, I will post it here for all of you to play with. Until then, checkout my other OpenCV articles or head over to my official Computer Vision website at http://www.learncomputervision.com.

Augmented Reality - 3D Sunglasses

Augmented Reality - 3D Cowboy Hat

Thank you for your interest in my site. If you find the information provided on this site useful, please consider making a donation to help continue development!

PayPal will open in a new tab.
$2.00
$5.00
Other

Related Posts

Tagged with:  

22 Responses to Augmented Reality Tests with OpenCV and C#

  1. aft1972 says:

    Hello

    Thanks for your work

    i am trying to build a fashion augmented reality application

    i used your code here:
    http://www.prodigyproductionsllc.com/articles/programming/augmented-reality-using-c-and-opencv/

    but i have 2 problems:
    1. the color of the photo you display on the chess board is mixed with the background color. how can i made the photo displayed as solid.

    2. when i put the chess board on my stomach i want the dress photo to cover all of my body centered at the chess board. is that available??

    Regards

    • LuCuS says:

      Sorry for the late reply. I have been out of the country and had very limited access to the internet.

      1. After line 119, you will need to add a new line with the following code. This will need to be placed inside of the “if (_option == 1) {” code block.

      Cv.Not(blank, blank);

      2. You can resize the overlayed image by adding “+ #” to the end of each line at lines 137 – 140 where “#” is whatever incremental size you want to add to increase the size of the overlayed image. You’ll have to play around with the numbers to get the size you’re looking for.

      As for the screenshots in the article above, I used the size of the face to determine the size of the image overlays. Since people have an average face size, it was easy to calculate how far the camera was from the face and then resize my 3D models according to the distance between the face and the camera.

      Unfortunately, I can’t provide you with much more information than that as I am actually in the process of designing a similar application for one of my customers which will act as a “virtual fitting room” and would be a conflict of interest with my employer if I shared too much information.

      • aft1972 says:

        Thank you very much

        I tried
        Cv.Not(blank, blank);

        It is working but the dress photo i have is with a transparency color around the dress. This transparency area is appeared in black. how can i make it transparent

        Regards

        • LuCuS says:

          Here is a method I use for working with transparent images.

          IplImage OverlayImage(IplImage src, IplImage overlay, CvPoint location, CvScalar S, CvScalar D)
          {
          for (int x = 0; x < overlay.Width; x++)
          {
          if (x + location.X >= src.Width) continue;
          for (int y = 0; y < overlay.Height; y++)
          {
          if (y + location.Y >= src.Height) continue;
          CvScalar source = Cv.Get2D(src, y + location.Y, x + location.X);
          CvScalar over = Cv.Get2D(overlay, y, x);
          CvScalar merged;

          /*
          merged.Val0 = (S.Val0 * source.Val0 + D.Val0 * over.Val0);
          merged.Val1 = (S.Val1 * source.Val1 + D.Val1 * over.Val1);
          merged.Val2 = (S.Val2 * source.Val2 + D.Val2 * over.Val2);
          merged.Val3 = (S.Val3 * source.Val3 + D.Val3 * over.Val3);
          */

          merged.Val0 = source.Val0 * over.Val0 / 255;
          merged.Val1 = source.Val1 * over.Val1 / 255;
          merged.Val2 = source.Val2 * over.Val2 / 255;
          merged.Val3 = source.Val3 * over.Val3 / 255;

          Cv.Set2D(src, y + location.Y, x + location.X, merged);
          }
          }

          return src;
          }

          And here is an example of how to use it:

          outImage = OverlayImage(img, overlayImg,
          new CvPoint(center.X - (overlayImg.Width / 2), center.Y - (overlayImg.Height / 2)),
          new CvScalar(1, 1, 1, 1), new CvScalar(0.1, 0.1, 0.1, 1));

          • aft1972 says:

            what is the purpose of this code??

            Regards

          • aft1972 says:

            can u give me more details about this code??
            i can not understand what it is exactly doing

          • LuCuS says:

            This code allows you to overlay an image (such as a PNG with transparency) over the top of another image or a video feed. You will pass it the image or video (frame) that you want to overlay another picture on top of, the overlay image itself, the coordinates for where you want the image to be displayed, and the min & max values of your transparency color(s).

      • aft1972 says:

        Also is it possible to paste here the lines 137 – 140 you mean??
        i think we have a different version from the source code

        Regards

        • LuCuS says:

          You will actually need to modify lines 132-140. The lines that look like this:

          p[0].X = corners[0].X;
          p[0].Y = corners[0].Y;
          p[1].X = corners[4].X;
          p[1].Y = corners[4].Y;

          p[2].X = corners[19].X;
          p[2].Y = corners[19].Y;
          p[3].X = corners[15].X;
          p[3].Y = corners[15].Y;

          will need to look something like this:

          p[0].X = corners[0].X - 20;
          p[0].Y = corners[0].Y - 20;
          p[1].X = corners[4].X - 20;
          p[1].Y = corners[4].Y - 20;

          p[2].X = corners[19].X + 20;
          p[2].Y = corners[19].Y + 20;
          p[3].X = corners[15].X + 20;
          p[3].Y = corners[15].Y + 20;

          You will need to mess around with the numbers to get it close to what you’re looking for. Instead of using hardcoded numbers though, I would replace them with a variable that gets calculated by the size of the detected chessboard so that the overlayed image resizes according to how far the chessboard is from the camera.

      • aft1972 says:

        Any way we are not building a commercial software it is an undergraduate project in MSA university

        Regards

  2. aft1972 says:

    Hello

    Is there any way to use this wrapper to get the gender and age of a person standing in front of the camera??

    Regards

    • LuCuS says:

      No. You will need to use some sort of classifier to handle that.

      • aft1972 says:

        can you give me more details??

        • LuCuS says:

          I have done something like this using both the EigenFaces and FisherFaces algorithms (both provided by OpenCV). Although EigenFaces was a little easier to implement, I got much better results using the FisherFaces approach. Since EigenFaces is a little easier to implement, I’ll quickly mention the steps needed for that. First, you will need to get a collection of images for each age range and gender you want to classify (eg. ages 1 – 18, 19 – 40, 41 – 75, 75+ and male & female). Then you will need to build an implementation of the Eigen Faces recognizer that allows you to train on the images for each age range & gender. Once you have your classifier trained, you can use any distance methods to compare your test image with your training images. I went with the Euclidean distance algorithm as it’s very simple to implement and gave me the best results.

          Unfortunately I can’t share any of the code for this at this time as I am working on this very thing for my current employer and it would be a conflict of interest to share code that directly relates to my current job. With that said, everything mentioned here is extremely simple to do and examples can be found all over the web.

          If you don’t want to use the methods I have mentioned here, you can always create Haar classifiers for each age range & gender and check if the new test image falls within any of the classifiers you have created for each age range & gender. You won’t get very good results this way, but it does work. Here is an article showing how to create your own Haar classifiers: http://www.prodigyproductionsllc.com/articles/programming/how-to-train-opencv-haar-classifiers/. In this article I show how to create classifiers for identifying BlackBerry cellphones, but you can use this same method using the images from your age ranges & genders image collections.

  3. rinks patel says:

    hi,sir

    can u send me full c# form file please ,because i facing some errors so please send me as soon as possible or source,goggles and hat…please as soon as possible

    • LuCuS says:

      Sorry, but I’ve chosen not to share the code for this yet. As for the 3D models, I cannot share those due to licensing issues. I am not the creator / owner of those models. Look around on the web and you can find all kinds of publicly available 3D models. That’s where I found the hat & glasses. You can also create your own models using any 3D modeling tool including the free Milkshape 3D (http://chumbalum.swissquake.ch/).

      • rinks patel says:

        hi,sir

        thank you sir for earlier regard ,i completely understand .i used real time face recognition using haarcascade for mark and opencv both together possible..?? i making my clg project…please as soon as possible

        • LuCuS says:

          You will begin with using a haarcascade to recognize the face. Once you have the face, you will use what’s called “good features to track” which will identify the points on the face (corners of eyes, nose, mouth, etc…). Next, you will use delaunay triangulation to identify which point on the face is which. By combining all of these, you will be able to identify the tilt & rotation of the face. Having the rotation of the face will tell you how to rotate / translate the 3D model and where to position it correctly on the face or head.

          • rinks patel says:

            Hii Sir !!!
            Thanks for warm reply to me. I have been already Face Recognizition(Haarcascade) ready made Project But, I want to use your augment reality Open CV, So kindly tell me. How could i patch this new feature(open Cv) in my project. if u no problem to give ur mail ID then please converse me on this given my mail address : rinks.patel14@gmail.com .
            please connect with me as sson as possible.

Leave a Reply