Detect Faces

One of the most amazing hidden features of the camera software is the ability to detect faces. We’ve seen that access to the pixels values enables us to interpret images, and make inferences about their content. Such computer vision algorithms have many applications in robotics, automation, and secu-rity. The Android face detection²⁸ API is designed to trigger an event when one or more faces are detected.

Facial recognition is an Android feature that uses complex computer vision algorithms to detect typical facial features, recognized by their shape and the position of a person’s eyes within the camera’s field of view. The Android device uses so-called Haar cascades²⁹ for the face recognition. The Camera app for instance uses this feature to set the focus of the camera on the eyes of a person when taking an photo. Face Unlock added to Ice Cream Sandwich uses face recognition to unlock your device. When you first activate Face Unlock (Security Settings→Face Unlock), you provide an image of your face and a PIN. The device remembers the shape and other characteristics of your face, and uses those metrics to compare it to a live camera image when you unlock the screen. Depending on the amount of available light, this feature works uncannily well.

Face detection is part of Android’s ^Camera class, exposed by KetaiCamera so we can us it within the camera apps we develop using the Ketai library. The information we receive about facial features includes the location of the ^leftEye(), the ^rightEye(), the ^mouth(), an individual ^id for each detected face, and a ^score of the confidence level for the detection of the face, ranging from ^1..100. The ability to detect facial features might come as a surprise when we use and expose it, however modern digital cameras use similar algorithms to auto-set the focus, and auto-correct red-eye effects.

The face finder sketch we are writing based on Android’s ^Face detection API is shown in action in Figure 23, Face Finder App, on page 132. For the sketch, we use the camera preview image and send it to the Face detector. It returns an array of faces to us, containing the metrics of individual facial features which we can use to draw a rectangle where a face is detected. We test the app on the device, point the Android camera to a webpage that displays the results of a Google Image search on the term "faces". This way, we can see how well the detection works when it has to respond to a variety of faces of different scales and quality. Let’s take a look:

28. http://developer.android.com/reference/android/hardware/Camera.Face.html 29. http://en.wikipedia.org/wiki/Haar-like_features

Figure 23—Face Finder App. The image illustrates Android’s Face Detector API, which here displays fourteen faces found by an image search engine. Side profiles and cropped

portraits are not recognized.

Camera/CameraFaceFinder/CameraFaceFinder.pde import ketai.camera.*;

import ketai.cv.facedetector.*;

KetaiCamera cam;

KetaiSimpleFace[] faces;

❶

boolean findFaces = false;

void setup() {

orientation(LANDSCAPE);

cam = new KetaiCamera(this, 640, 480, 24);

rectMode(CENTER);

❷

stroke(0, 255, 0);

noFill();

❸

}

void draw() { background(0);

if (cam != null) {

image(cam, 0, 0, 640, 480);

if (findFaces)

❹

{

faces = KetaiFaceDetector.findFaces(cam, 20);

❺

Chapter 5. Using Android Cameras

•

for (int i=0; i < faces.length; i++)

text("Faces found: " + faces.length, 680, height/2);

❾

Let’s take a look at the face finder methods used by the sketch:

❶ Create array to store the list of faces found. It contains the x and y location of each face and the distance between its eyes

❷ Center the rectangles that mark found faces around their center point

❸ Turn off the fill color for the green rectangle markers, so we can see though them

❹ Check the boolean that let’s us turn the face finder on and off

❺ Call the findFaces() in the ^FaceFinder class, with the two parameters for the image input (^cam), and the maximum number of faces to be found (²⁰)

❻ Parse the results returned from the faces array. The array length varies depending on how may faces are found, so we check how often to iterate through the array by testing faces.length with the ^for loop

❼ Draw a rectangle based on the returned face location PVector. Use ^.x() and .y() to access the horizontal and vertical position of the face location.

❽ Use double eye distance to draw approximate rectangle marking the detected face

❾ Display the total number of faces found, a maximum of 20 can be found based on our findFaces() settings

Let’s give it a try.

Run the App

Run the app and set the device aside. Now go to your PC and do a Google image search on the word "face". Pick up the Android and aim the camera at the PC display. Google displays a grid of images showing a wide range of person’s faces at different exposures and angles. Now tap the screen to start face detection. You immediately experience a performance hit caused by the face detection algorithm. We’ve instructed the findFaces() to extract up to twenty faces from the camera preview.

Once the camera has a clear and steady shot at the faces on the PC display, you can see on the Android screen green rectangles overlaid onto the detected areas, as illustrated in Figure 23, Face Finder App, on page 132. Overall, it does a pretty good job. When portraits are cropped or only show faces in a profile, the algorithm doesn’t consider it a face. To confirm this rule, do a Google search on the term "face profile" and see what happens. Finally, see what "cartoon face" will produce. Using these different search strings help us understand how the algorithm requires certain facial features to interpret a certain pixel area as a "face".

Let’s move on to the detection of moving human subjects. Use setCameraID(1) just before cam.start(); in setup() to switch to the front-facing camera. Run the app again, and test the face detection algorithm on your own face. You should observe that the face detection feature begins to work as soon as you face the camera. You need to keep enough distance so the face doesn’t appear cropped in the camera preview. If you turn your head to present a profile to the camera , your face won’t be detected anymore, because the camera can’t "see" both your eyes.

We haven’t looked deeply into what the Face API does exactly to extract faces from a list of pixel values, and in this case, we don’t need to. Android provides us with a list of faces, the midpoint between the eyes, and their distance.

Edge detection and decision trees are the concern of API. Clearly, this feature, which ships with all current Android devices, can be used for different pur-poses.

Unlike social media sites which employ face detection algorithms to match a person or identity with an image, the Android is not concerned about that. If we start up face detection in our app, the Android OS will trigger a face event Chapter 5. Using Android Cameras

•

when it "sees" a face, whether or not it knows the identity of that person. For some of your apps, it can be relevant to know whether a person is looking at the screen, or not.

Now that you are aware of this feature, it’s up to you to use it, or look at your device from now on with the level of scrutiny this feature calls for. The face detection project is a good example to understand why we need to ask for permission to use the ^CAMERA (Section 4.4, Setting Sketch Permissions, on page 83). If we do, the user is prompted to grant or deny this request. Once granted, the app will retain permission certificate to use the camera, and we won’t be prompted any more. In a section to be written, we’ll used the face detection feature to rotate a 3D object based on how we look at the screen.

It is one example where the face detection API serves as an interactive user interface within a 3D environment.

5.9 Wrapping Up

In this chapter, you’ve learned how to work with the cameras on Android devices. You’ve also learned how to display the images the cameras acquire, how to save them, and how to work with them when they’re stored in memory.

You’re now able to manipulate images down to the pixel level, and use the camera to detect colored objects in motion and display their paths on the screen. You’ve also learned how to activate and use Android’s face recognition feature.

This completes our investigation of a diverse range of sensors found on Android devices. You know how to interact with their touch screens, determine their orientation and bearing, their motion and geographic location. You can also take pictures with the Android, and start to make sense of what the device is seeing. You’re ready now to move on to the second part of this book, where we’ll learn how to network the Android with PCs and other mobile devices, and work with large amounts of data.

Part III