Ever since receiving my new Google Glass a few weeks ago, my inbox has been flooded from people asking if I have built any computer vision apps yet for Glass. The answer to those questions is “yes” and I plan on posting some articles and videos about them very soon. Until then, I thought it would be a good idea to post some articles showing how to get started with computer vision development on Google Glass using OpenCV for Android. Since there are several steps involved with building useful computer vision apps for Glass, I will be breaking up the steps involved into multiple articles. In this article, I will walk you thru the steps required to get your first OpenCV app installed and running on your Glass. While I’m at it, I will also show you how to install and run a simple face detection app. In a future article, I will show you how to do more such as applying filters and even a little bit of augmented reality.
Java Development Kit (JDK)
Just like developing for any other Android app, the first thing you will need for developing on Glass is the Java Development Kit (JDK) from Oracle. In this article, I will be working with JDK7. So, I recommend you use the same to keep from having any unexpected issues. You can download JDK7 for free from http://www.oracle.com/technetwork/java/javase/downloads/index.html. When you get there, you will see a section labeled “Java Platform, Standard Edition”. Underneath it, click the blue button under “JDK”. This will take you to a page where you must read and accept the Oracle Binary Code License Agreement for Java SE. Make sure you click the radio button to accept the agreement. Otherwise you will not be able to download the software.
With the Accept License Agreement button selected, click on the link next to Windows x64 (jdk-7u45-windows-x64.exe). For the purposes of this series of articles, I will be working with 64 bit Windows 7. In the event that you are using something other than Windows, you will need to select the JDK version that is compatible with your operating system and processor. At the time of writing this article, the latest version is 7u45. Depending on when you are reading this article, the version will most likely be different. Therefore, you will see a different link to download from. Once you have the JDK downloaded, go ahead and run it and follow the on-screen instructions until you have finished installing Java7.
Android Development Tools (ADT) and SDK
After you have installed Java, the next thing you will need for creating your Android apps is an IDE (integrated development environment) such as Eclipse. If you’re not familiar with Eclipse, it is an IDE that includes everything you need for almost any programming language. I’ve used Eclipse for developing apps I’ve written in C, C++, Perl, PHP, Python, Flex, and many other languages. But, for the purposes of this article, we will be using Eclipse for working with Java. In most of my articles, I recommend downloading native Eclipse or Eclipse for J2EE development. In this article, we will be downloading & installing an extended version of the Eclipse IDE that includes the built-in ADT (Android Developer Tools). The extended version of Eclipse that we will be using for our Glass development already includes the Android SDK. If you’re interested in adding the Android SDK to an existing installation of Eclipse, feel free to checkout this article.
To download the Android prepared Eclipse, head over to https://developer.android.com/sdk/index.html and click on the big blue button that reads “Download the SDK ADT Bundle for Windows”. This will display the Android SDK terms & conditions. If you agree to the terms & conditions, check the box at the bottom that reads “I have read and agree with the above terms and conditions”. Next, check the radio button next to the version you want to download and click the blue “Download the SDK ADT Bundle for Windows” button. Since I’m working with 64-bit as mentioned above, I will select the “64-bit” radio button.
Once you click the download button, it will begin downloading the Android prepared version of Eclipse (to be known as ADT hence forth). At the time of writing this article, the file that gets downloaded is called “adt-bundle-windows-x86_64-20131030.zip”. Once you have downloaded ADT, the installation is extremely simple. The only thing you have to do to install it is unzip it to your filesystem. Any extractor will work, including the one that comes built into Windows 7. Before you run ADT, I would recommend that you move the unzipped folder to a more permanent location such as the root of your C:\. After you have unzipped ADT, you can run it by going into the unzipped folder and double-clicking the “eclipse.exe” file. Depending on how often you will be developing for Glass (or Android in general), I would recommend right-clicking on the “eclipse.exe” file and choosing “Send To > Desktop (create shortcut)”. This will create a shortcut on your Desktop, making it easier to get to going forward.
Now that you have the Android Developer Tools installed, start it up by double-clicking the ADT shortcut you created on your Desktop and select a location for your Workspace. The workspace is simply a folder on your file system that contains your Android projects. So, it doesn’t really matter where you create your Workspace. Eclipse will remember its location for you. When opening ADT for the first time, you will be presented with a welcome screen. You can close it as we will not be using it in this article. Next, click on “Window” from the menu bar and scroll down to & click on “Android SDK Manager”.
This will launch a separate window that allows you to select & download the components you will be working with. Since Google Glass only supports API 15 currently, you will need to check the boxes under “Android 4.0.3 (API15)” for “SDK Platform” and “Glass Development Kit Sneak Peek”. You should also go ahead and install any of the other recommended (already checked) items as well. However, I will warn you that downloading and installing several SDKs can take a fairly long time. Also, we will not be using any of the other SDKs during this series of articles. So, it’s not necessary to install them at this time. If you decide to install them at a later time, you can always repeat these steps to get back to the SDK Manager for installation. However, unless you manually installed the USB drivers as I explained in yesterday’s article, you should also check the “Google USB Driver” item located under the “Extras” folder so that you can push your apps from ADT onto Glass. With the currently needed items checked, click the “Install X packages…” button in the lower-right corner where “X” is the number of packages you are installing (should be 2 unless you have decided to install other packages at this time). Note: You will probably be asked to close ADT during the installation so that the installer can write to the SDK directory.
CDT & NDK
After the SDK Manager has finished downloading and installing the required packages, restart ADT. Once ADT is reopened back to your Workspace, click on “Help > Install new software…”. This will launch another window. In the upper-right corner, click the “Add…” button. In the “Name” field, enter “CDT” then copy and paste the following URL into the “Location” field and click the “OK” button. “http://download.eclipse.org/tools/cdt/releases/indigo“. Follow the on-screen instructions until CDT has been installed. Then, repeat the same steps, but using “NDK” for the “Name” field and the following URL for the “Location” field. “https://dl-ssl.google.com/android/eclipse“.
After you have the CDT & NDK plugins installed, point your browser to http://developer.android.com/tools/sdk/ndk/index.html. From there, select the “Package” for your environment. Since I have been working with 64-bit Windows throughout this article, I will click on the “android-ndk-r9c-windows-x86_64.zip” link. The file is ~0.5GB. So, it’ll take a little while to download. Once it has downloaded, you can install it just like you did ADT simply by unzipping it to your file system. Just like ADT, it’s a good idea to move the unzipped NDK folder to a more permanent location on your file system. Just make sure you put it somewhere you can remember easily as you’ll be needing to point to it in an environment variable. I moved my NDK folder to “C:\” and renamed the folder to “AndroidNDK”.
NDKROOT Environment Variable
To create an environment variable for NDK, right-click on “My Computer” and select “Properties”. Next, click on “Advanced system settings” on the left. This will popup the “System Properties” dialog window which should be on the “Advanced” tab. From there, click the “Environment Variables” button. Click the “New” button under the “System variables” list. For the “Variable name” field, enter “NDKROOT”. For the “Variable value” field, enter the full path to the NDK folder you extracted earlier. After you have added the “NDKROOT” environment variable, click the “OK” button on each window and close any remaining windows until you get back to to ADT.
Note: You’ll probably need to restart ADT (Eclipse) after adding the NDKROOT environment variable.
The last thing you will need to download and install will be the OpenCV4Android SDK which you can find at http://sourceforge.net/projects/opencvlibrary/files/opencv-android/. Instead of clicking the “exe” file listed at the top of SourceForge, click on the latest version number and select the “zip” file. The current version is 2.4.7. So, I will need to click on OpenCV-220.127.116.11-android-sdk.zip. Just like everything else so far, all you need to do to install the OpenCV4Android SDK is unzip it. And, as usual, you’ll want to make sure to move it to a location on your file system that is easily remembered as you’ll be referring back to it a lot.
Once you have everything installed (unzipped), you are ready to create your first OpenCV app for Google Glass. To do that, open ADT and enter your Workspace. Then, click on “File > Import”. Drop down the arrow next to “Android” and select “Existing Android Code Into Workspace” and click the “Next” button. Click the “Browse…” button on the next window and navigate to the unzipped OpenCV4Android folder and click “OK”. You should now see a list of samples as well as an item for “sdk\java”. For now, you will only need to select the “sdk\java” and the “OpenCV Sample – face-detection” items followed by clicking the “Finish” button.
After you have imported those 2 projects, you might see some errors in your Workspace which will prevent you from being able to do a clean build. If you do, you will need to right-click on the “OpenCV Sample – face-detection” project and select “Properties”. Click on the “Android” item on the left and verify that your reference to the “OpenCV Library” is correct.
You’ll also need to click on “C/C++ Build” and verify that your “Build command” and “Build directory” are correct.
Installing OpenCV on Google Glass
Before you can run your face detect app on Google Glass, you will first need to install the OpenCV library on it. Since OpenCV is not provided in Google Play, you will have to install it manually. Before you can do that, you will need to add a new environment variable that points to the location where you unzipped your Android SDK. You can follow the same steps as provided above for setting up the NDKROOT environment variable, but use “ANDROID_HOME” for the variable name and your Android SDK folder as the variable value. After you have added the ANDROID_HOME environment variable, you will need to append it to your PATH variable followed by “\platform-tools” and again for “\tools”. If you named your Android Home environment variable “ANDROID_HOME”, you can copy the following line and paste it at the end of your “PATH” variable.
Next, open a command prompt and navigate to the location where you unzipped the OpenCV4Android SDK. There you will need to run the following command which will install the OpenCV APK on your Google Glass (assuming you have Glass connected to your computer and already configured). If you have any problems getting your Glass connected to your computer, you should checkout this article which explains how to configure your Google Glass with Windows 7.
adb install apk/OpenCV_2.4.7_Manager_2.14_armv7a-neon.apk
Running Face Detect on Google Glass
Before you are ready to run the OpenCV sample app, there is one change you will need to make to the code before it will work correctly on your Google Glass. If you run the app as it is right now, you will see a bunch of lines zig-zagging across your Glass display. This is happening because the face recognition app is not properly scaled to work with the Glass’ resolution. Because of this, the pixels at the end of each line tend to wrap to the following lines which give it the zig-zagging lines effect. To prevent this from happening, you will need to manually specify the size of your Glass display in the “activity_main.xml” layout. All you have to do is give a layout_width of “640dp” and a layout_height of “360dp” as shown below. Make sure you add these values for the LinearLayout and the JavaCameraView.
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="640dp" android:layout_height="360dp" tools:context=".MainActivity" > <org.opencv.android.JavaCameraView android:id="@+id/fd_activity_surface_view" android:layout_width="640dp" android:layout_height="360dp" /> </LinearLayout>
Once you have the Glass resolution specified, all of your Workspace errors corrected, and the OpenCV APK installed on your Google Glass, you are now ready to launch the face detection app on your Glass for the first time. If you don’t have Google Glass, you can always run your app using your other Android powered devices or using the Android Emulator that is included in the Android SDK as explained in this article. If you do have Google Glass and have it already setup & configured with your computer, connect it to your computer using the provided USB cable. Then, right-click on the “OpenCV Sample – face-detection” project and select “Run As > Android Application”. There you should see your Google Glass listed in the top box for “Choose a running Android device”. Select your Glass from the list and click the “OK” button. This will push the app to your Glass where it will be launched for the first time.
PayPal will open in a new tab.